-- Logs begin at Thu 2022-08-25 23:52:02 UTC, end at Fri 2022-08-26 01:16:53 UTC. -- Aug 26 00:33:35.586155 dv-cinder-835384-25-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 26 00:33:35.593384 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 26 00:33:35.595364 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 26 00:33:35 2022] *** Aug 26 00:33:35.595364 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 26 00:33:35.595364 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 26 00:33:35.595364 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: nodename: dv-cinder-835384-25-iscsi Aug 26 00:33:35.595364 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: machine: x86_64 Aug 26 00:33:35.595364 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: clock source: unix Aug 26 00:33:35.595364 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: pcre jit disabled Aug 26 00:33:35.595910 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: detected number of CPU cores: 4 Aug 26 00:33:35.595910 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: current working directory: / Aug 26 00:33:35.595910 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: detected binary path: /usr/bin/uwsgi-core Aug 26 00:33:35.595910 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: your processes number limit is 47680 Aug 26 00:33:35.595910 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: your memory page size is 4096 bytes Aug 26 00:33:35.595910 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: detected max file descriptor number: 2048 Aug 26 00:33:35.595910 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: lock engine: pthread robust mutexes Aug 26 00:33:35.595910 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: thunder lock: enabled Aug 26 00:33:35.595910 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 26 00:33:35.595910 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 26 00:33:35.636157 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: Python main interpreter initialized at 0x55ef291d29c0 Aug 26 00:33:35.636157 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: python threads support enabled Aug 26 00:33:35.636157 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: your server socket listen backlog is limited to 100 connections Aug 26 00:33:35.636157 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: your mercy for graceful operations on workers is 90 seconds Aug 26 00:33:35.636561 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: mapped 403077 bytes (393 KB) for 2 cores Aug 26 00:33:35.636642 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: *** Operational MODE: preforking *** Aug 26 00:33:35.636904 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: *** uWSGI is running in multiple interpreter mode *** Aug 26 00:33:35.636904 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: spawned uWSGI master process (pid: 88242) Aug 26 00:33:35.637106 dv-cinder-835384-25-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 26 00:33:35.638307 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: spawned uWSGI worker 1 (pid: 88253, cores: 1) Aug 26 00:33:35.638307 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: spawned uWSGI worker 2 (pid: 88254, cores: 1) Aug 26 00:33:35.638307 dv-cinder-835384-25-iscsi devstack@keystone.service[88242]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 26 00:33:39.945457 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 00:33:39.945457 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: from cryptography.utils import int_from_bytes Aug 26 00:33:39.946241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 00:33:39.946241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: from cryptography.utils import int_from_bytes Aug 26 00:33:39.946759 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 00:33:39.946759 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: from cryptography.utils import int_from_bytes Aug 26 00:33:39.947228 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 00:33:39.947228 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: from cryptography.utils import int_from_bytes Aug 26 00:33:39.998647 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 26 00:33:39.998830 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 26 00:33:39.999016 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] command line args: [] {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 26 00:33:39.999290 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 26 00:33:39.999469 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ================================================================================ {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 26 00:33:39.999730 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] admin_token = **** {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:39.999980 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] config_dir = [] {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.000167 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.000349 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] config_source = [] {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.000556 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] debug = True {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.000759 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.002158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.002158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.002158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.002158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.002158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] list_limit = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.002158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] log_config_append = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.002610 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.002610 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] log_dir = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.002746 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] log_file = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.002904 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.003151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.003399 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.003584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 26 00:33:40.003705 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.004429 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 26 00:33:40.004707 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.004789 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] command line args: [] {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 26 00:33:40.004851 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.004911 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 26 00:33:40.004969 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.005034 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ================================================================================ {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 26 00:33:40.005108 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.005176 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] admin_token = **** {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.005242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.005475 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] config_dir = [] {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.005580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.005663 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.005746 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.005827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] config_source = [] {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.005900 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] debug = True {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.005971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.006100 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.006239 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.006335 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.006442 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.006514 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.006651 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.006743 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.006977 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.007143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.59/identity {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.007232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] list_limit = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.007319 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] publish_errors = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.007393 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] log_config_append = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.007522 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.007602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.007696 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] log_dir = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.007775 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.007861 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] log_file = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.007938 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.008088 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.008183 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.008330 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.008410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.008598 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.008682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] standard_threads = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.008858 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.008944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.009021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.009148 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.009269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.009422 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.009507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] use_journal = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.009614 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] use_json = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.009699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.009774 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] use_stderr = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.009912 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.009997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] use_syslog = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.010123 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.010204 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.010353 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.010461 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.010584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.010665 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.010813 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.010892 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.011057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.011149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.011304 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.011391 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.011578 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.011658 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.011828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.012061 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.59/identity {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.012150 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.012231 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] publish_errors = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.012316 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.012452 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.012610 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.012695 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.012859 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.012950 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.013101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.013178 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.013286 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.013368 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.013526 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.013606 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] standard_threads = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.013752 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.013834 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.013992 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.014081 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.014155 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.014227 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.014329 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] use_journal = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.014487 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.014567 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] use_json = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.014658 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] use_stderr = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.014736 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.014831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] use_syslog = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.014979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.015067 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 00:33:40.015221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.015302 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.015481 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.015572 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.015748 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.015827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.015989 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.016068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.016235 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.016326 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.016484 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.016567 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.016722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.016814 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.016955 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.017043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.017192 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.017271 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.017434 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.017514 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.017677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.017765 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.017907 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.017985 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.018140 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.018219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.018370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.018533 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.018610 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.018700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.018847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.018928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.019081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.019169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.019337 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.019422 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.019579 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.019665 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.019824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.019915 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.020055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.020133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.020288 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.020374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.020538 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.020659 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.020766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.020865 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.020994 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.021110 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.021244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.021360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.021467 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.021574 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.021694 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.021773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.021925 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.022020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.022167 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.022247 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.022406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.022526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.022633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.022767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.022847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.022908 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.023093 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.023181 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.023332 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.023415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.023564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.023654 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.023800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.023920 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.024055 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.024149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.024263 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.024343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.024505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.024593 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.024719 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.024846 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.024994 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.025074 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.025141 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.025276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.025367 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.025497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.025584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.025704 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.025824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.025947 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.026052 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.026141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.026270 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.026350 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.026502 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.026620 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.026755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.026864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.026960 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.027118 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.027200 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.027369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.027457 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.027614 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.027710 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.027849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.027929 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.028085 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.028167 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.028320 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.028413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.028911 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.029006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.029081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.029152 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.029309 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.029408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.029545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.029626 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.029780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.029861 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.030012 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.030102 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.030250 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.030333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.030484 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.030565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.030714 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.030802 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.030951 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.031032 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.031187 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.031267 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.031443 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.031536 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.031679 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.031761 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.031925 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.032012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.032166 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.032250 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.032399 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.032531 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.032655 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.032736 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.032891 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.032972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.033130 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.033212 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.033365 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.033445 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.033597 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.033682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.033832 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.033915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.034065 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.034145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.034299 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.034378 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.034533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.034615 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.034768 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.034848 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.035027 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.035110 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.035264 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.035354 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.035499 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.035580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.035757 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.035838 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.035996 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.036078 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.036231 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] auth.external = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.036317 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.036466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.036567 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.036725 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.036810 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.036964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.037044 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.037200 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.037287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] auth.password = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.037433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.037516 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] auth.token = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.037667 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.037747 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.037896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.037976 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.038124 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.038207 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.038357 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.038446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.038573 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.038696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.038781 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.038862 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.039008 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.039092 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.039241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.039358 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] credential.caching = True {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.039487 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.039572 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.039720 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.039803 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.039972 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.040060 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.040226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.040306 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.040463 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.040591 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.040712 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] auth.external = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.040796 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.040940 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.041022 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.041204 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.041286 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.041447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.041529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.041673 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] auth.password = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.041753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.041955 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] auth.token = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.042054 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.042130 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.042204 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.042344 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.042473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.042588 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.042722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.042803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.042983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.043065 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.043223 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] federation.caching = True {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.043306 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.043464 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.043546 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.043712 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] credential.caching = True {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.043800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.043953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.044031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.044278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.044369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.044434 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.044526 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.044682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.044764 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.044917 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.044999 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.045152 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.045233 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.045398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.045481 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.045637 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.045720 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.045871 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.045951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.046112 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] identity.caching = True {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.046194 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.046364 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.046446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.046615 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.046699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.046857 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.046941 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.047095 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.047176 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.047346 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.047425 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.047576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.047657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] federation.caching = True {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.047810 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.047896 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.048055 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.048138 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.048301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.048384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.048557 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.048644 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.048793 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.048878 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.049026 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.049108 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.049270 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.049350 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.049501 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.049582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.049752 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.049836 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.049992 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.050074 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.050236 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.050317 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.050471 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] identity.caching = True {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.051051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.051245 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.051327 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.051492 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.051574 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.051727 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.051809 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.051960 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.052048 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.052194 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.052274 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.052421 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.052516 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.052681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.052762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.052927 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.053009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.053178 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.053262 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.053408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.053489 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.053635 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.053724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.053863 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.053952 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.054104 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.054185 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.054348 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.054437 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.054592 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.054674 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.054824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.054904 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.055067 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.055157 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.055333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.055438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.055570 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.055688 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.055806 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.055935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.056019 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.056186 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.056267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.056430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.056532 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.056700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.056785 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.056937 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.057017 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.057174 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.057256 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.057420 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.057503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.057646 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.057726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.057890 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.057971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.058133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.058217 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.058365 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.058453 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.058602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.058683 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.058842 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.058924 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.059069 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.059150 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.user = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.059305 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.059387 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.059532 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.059615 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.059765 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.059970 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.060051 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.060126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.060233 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.060318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.060466 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.060622 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.060723 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.060805 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.060967 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.061050 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.061206 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.061289 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.061433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.061513 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.061660 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.061744 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.061891 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.061971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.062138 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.062220 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.062369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.062453 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.062600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.062682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.062832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.062914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.063060 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.user = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.063143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.063296 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.063378 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.063542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.063665 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.063783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.063898 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.064023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.064145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.064264 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.064401 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.064581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.064689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.064785 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.064880 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.065002 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.065088 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.065243 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.065316 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.065501 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.065585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.065734 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.065816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.065974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.066057 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.066260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.066965 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.067055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.067120 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.067253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.067337 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.067497 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.067579 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.067741 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.067895 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.067975 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.068049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.068200 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.068283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.068436 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.068550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] resource.caching = True {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.068695 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.068779 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.068931 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.069013 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.069165 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.069255 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.069408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.069491 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.069653 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.069734 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.069892 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.069982 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.070144 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.070226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.070373 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.070456 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.070614 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.070704 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.070845 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.070926 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] role.caching = True {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.071079 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.071163 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] role.driver = sql {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.071316 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.071407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.071558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.071640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.071797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.071880 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.072027 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.072117 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.072262 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.072344 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.072526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] resource.caching = True {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.072614 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.072770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.072865 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.073016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.073098 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.073251 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.073333 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.073488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.073582 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.073733 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.073815 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.073969 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.074050 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.074213 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.074303 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.074448 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.074530 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.074682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.074850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.074931 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] role.caching = True {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.075091 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.075175 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] role.driver = sql {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.075344 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.075425 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.075596 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.075676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.075842 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.075925 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.076087 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.076169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.076321 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.076402 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.076586 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.076670 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.076833 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.076914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.077078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.077173 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.077312 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.077394 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.077541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.077621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.077773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.077862 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.078016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.078098 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.078260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.078343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.078504 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.078595 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.078742 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.078899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.078981 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.079132 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.079215 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.079384 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.079468 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] token.caching = True {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.079632 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.079713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.079877 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.079959 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.080120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.080202 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.080347 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.080434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.080621 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.080705 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.080869 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.080953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.081103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.081184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.081332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.081414 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.081561 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.081644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.081807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.081892 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.082060 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.082142 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.082296 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.082379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.082545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.082626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.082776 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.082857 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.083009 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.083092 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.083237 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.083329 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.083472 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] token.caching = True {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.083554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.083706 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.083795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] database.connection = **** {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.083951 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.084033 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.084185 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.084266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.084438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.084552 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.084710 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.084793 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.084946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.085028 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.085183 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.085273 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.085422 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.085503 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.085659 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.085741 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.085893 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.085982 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.086127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.086209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.086359 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.086440 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.086594 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.086683 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.086847 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.086929 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.087077 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.087167 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.087320 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.087399 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.087543 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.087626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.087788 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.087870 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] database.connection = **** {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.088026 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.088107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.088190 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88254) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 26 00:33:40.088256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.088509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.088752 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.088986 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.089226 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.089459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.089693 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.089934 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.090166 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.090409 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.090643 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.090889 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.091148 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.091366 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.091593 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.091831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.092062 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 00:33:40.092228 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88253) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 26 00:33:40.118165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.118397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.118623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.121032 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.121266 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.121489 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.127097 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.130329 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.155848 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.155990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.156089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.156204 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.209326 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.209788 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.257721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.257953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.258607 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.258847 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.420358 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.420723 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.420822 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.421048 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.421268 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.421488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.422461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.422700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.422924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.423145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.423363 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.423579 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.450477 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.450615 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.450720 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.450811 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.450952 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.451057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.451182 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.451269 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.451407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.451492 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.451627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.451714 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.451844 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.451928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.466369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.466476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.482555 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.482681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.482787 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.482874 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.483017 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.483119 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.483246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.483332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.483460 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.483544 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.483677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.483764 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.483892 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88253) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.483978 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88254) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 00:33:40.500846 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.501000 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.501132 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 00:33:40.501480 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 00:33:40.501480 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.502808 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.502936 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 00:33:40.503054 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 00:33:40.503054 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.503256 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.503359 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.503467 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.503569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.504177 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.504307 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.504427 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.512851 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.513019 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.513387 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.513387 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.513533 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.513731 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.513816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.514052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.521370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88253) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.521708 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88254) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.524275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88253) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.524582 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.524681 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88254) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.524866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.524988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.525235 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.525522 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.525657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.525808 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.526082 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.526362 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.526461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.526638 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.526971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.527088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.527302 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.527579 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.527754 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.528157 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.528960 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.529380 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.529787 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.550800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.552289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88254) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.552609 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.552720 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.554202 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88253) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.554488 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.557721 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88254) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.558012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.558304 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.559462 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88253) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.559739 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.560023 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.564754 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88254) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.565047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.566947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88253) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.567232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.570300 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88254) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.570740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.571189 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.571330 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88253) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.571627 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.571892 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.571997 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.572178 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.572454 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.572756 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.572977 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.573130 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.573270 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.573868 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.574290 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.574709 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.591105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.591390 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.593540 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.593988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.595341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88253) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.598153 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88254) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.598609 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.598907 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.601464 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.601769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.603104 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88253) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.606026 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88254) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.606303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88253) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.609203 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88254) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.609588 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.609888 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.610171 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.610601 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.611055 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.611347 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.612823 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.613284 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.613739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.614185 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.614622 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.614927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.627325 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88253) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.628068 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.628068 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.628221 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.631577 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88254) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.631875 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.632171 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.632445 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.635558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.637122 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.637402 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.639405 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.640084 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88253) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.640372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.640705 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.640990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.641086 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.641270 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.641368 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.643939 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88254) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.644229 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.645110 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.645402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.645685 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.653173 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88253) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.653461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.653992 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.654274 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.654557 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.654839 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.655115 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.657635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88254) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.657926 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.658215 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.658502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.658791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.659074 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.659357 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.670542 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88253) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.673582 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88253) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.674980 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88254) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.676530 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.677974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.679447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88254) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.679535 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88253) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.679708 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.680146 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.681059 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.682531 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.684028 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88254) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.684314 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.684628 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.686669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.688117 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88253) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.691014 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.691796 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.691904 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.692110 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.692398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.693326 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88254) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.696237 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.696627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.696924 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.697210 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.700425 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88253) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.700751 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.701047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.705264 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88254) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.705576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.705869 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.708539 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88253) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.708976 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.709260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.709539 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.709813 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.710096 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.710374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.710650 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.710934 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.711220 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.711494 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.711766 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.712030 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.712289 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88253) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.713721 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88254) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 00:33:40.714016 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.714300 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.714580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.714870 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.715169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.715450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.715725 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.716021 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.716304 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.722958 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.722958 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.722958 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.722958 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88254) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 00:33:40.747298 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55ef291d29c0 pid: 88253 (default app) Aug 26 00:33:40.750952 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178e9b3-3e95-4180-b85d-62824ff31edd None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:40.751248 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178e9b3-3e95-4180-b85d-62824ff31edd None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:40.751519 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178e9b3-3e95-4180-b85d-62824ff31edd None None] PATH_INFO: `/v3` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:40.752526 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55ef291d29c0 pid: 88254 (default app) Aug 26 00:33:40.755943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1/1] 10.222.0.59 () {50 vars in 791 bytes} [Fri Aug 26 00:33:40 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:33:48.512492 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb3745-a81a-42cc-8d46-5b8b93f4257c None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:48.513069 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb3745-a81a-42cc-8d46-5b8b93f4257c None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:48.513069 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb3745-a81a-42cc-8d46-5b8b93f4257c None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:48.517282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1/2] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:33:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:33:48.523112 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb5dc3-1afe-4019-830c-988edb72438e None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:48.523534 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb5dc3-1afe-4019-830c-988edb72438e None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:48.523668 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb5dc3-1afe-4019-830c-988edb72438e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:48.593339 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: INFO dbcounter [None req-f0fb5dc3-1afe-4019-830c-988edb72438e None None] Registered counter for database keystone Aug 26 00:33:48.614600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG oslo_db.sqlalchemy.engines [None req-f0fb5dc3-1afe-4019-830c-988edb72438e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88253) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 26 00:33:48.620774 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writer thread running {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 26 00:33:48.812794 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG passlib.handlers.bcrypt [None req-f0fb5dc3-1afe-4019-830c-988edb72438e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=88253) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 26 00:33:48.812794 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG passlib.handlers.bcrypt [None req-f0fb5dc3-1afe-4019-830c-988edb72438e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88253) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 26 00:33:48.855369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-f0fb5dc3-1afe-4019-830c-988edb72438e None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:33:48.856997 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-f0fb5dc3-1afe-4019-830c-988edb72438e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:33:48.984755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 2/3] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:33:48 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 26 00:33:49.470818 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb3745-a81a-42cc-8d46-5b8b93f4257c None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:33:49.540422 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: INFO dbcounter [None req-70bb3745-a81a-42cc-8d46-5b8b93f4257c None None] Registered counter for database keystone Aug 26 00:33:49.560006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG oslo_db.sqlalchemy.engines [None req-70bb3745-a81a-42cc-8d46-5b8b93f4257c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88254) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 26 00:33:49.567188 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writer thread running {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 26 00:33:49.688450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb3745-a81a-42cc-8d46-5b8b93f4257c None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:33:49.693420 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc4d8a-608d-44df-a28b-5e3df065c484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:33:49.694061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc4d8a-608d-44df-a28b-5e3df065c484 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:49.694345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc4d8a-608d-44df-a28b-5e3df065c484 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:49.694621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc4d8a-608d-44df-a28b-5e3df065c484 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:49.702902 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cc4d8a-608d-44df-a28b-5e3df065c484 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:33:49.754903 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: WARNING keystone.common.rbac_enforcer.enforcer [None req-00cc4d8a-608d-44df-a28b-5e3df065c484 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 26 00:33:49.755249 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cc4d8a-608d-44df-a28b-5e3df065c484 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:33:49.764845 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 2/4] 10.222.0.59 () {58 vars in 1324 bytes} [Fri Aug 26 00:33:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 00:33:49.770811 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb5dc3-1afe-4019-830c-988edb72438e None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:33:49.785494 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb5dc3-1afe-4019-830c-988edb72438e None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:33:49.789997 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6a2b9-833b-4334-9f40-d40ef50ac922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:33:49.791099 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6a2b9-833b-4334-9f40-d40ef50ac922 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:49.791383 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6a2b9-833b-4334-9f40-d40ef50ac922 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:49.791655 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6a2b9-833b-4334-9f40-d40ef50ac922 admin admin] PATH_INFO: `/v3/projects/86ade54a72a44599b7a6313f909a3551` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:49.793235 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6a2b9-833b-4334-9f40-d40ef50ac922 admin admin] RBAC: Authorizing `identity:get_project(project_id=86ade54a72a44599b7a6313f909a3551)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:33:49.845411 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d0d6a2b9-833b-4334-9f40-d40ef50ac922 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 26 00:33:49.845759 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6a2b9-833b-4334-9f40-d40ef50ac922 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:33:49.850265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 3/5] 10.222.0.59 () {56 vars in 1207 bytes} [Fri Aug 26 00:33:49 2022] GET /identity/v3/projects/86ade54a72a44599b7a6313f909a3551 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:33:51.731517 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97197954-c9cc-4d4e-b51f-4573c70fea35 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:51.732023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97197954-c9cc-4d4e-b51f-4573c70fea35 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:51.732023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97197954-c9cc-4d4e-b51f-4573c70fea35 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:51.736066 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 3/6] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:33:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:33:51.741735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c55d6-36fb-4f5b-a0a3-f22eb80c80ae None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:51.742119 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c55d6-36fb-4f5b-a0a3-f22eb80c80ae None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:51.742302 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c55d6-36fb-4f5b-a0a3-f22eb80c80ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:51.762219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-274c55d6-36fb-4f5b-a0a3-f22eb80c80ae None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:33:51.763736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-274c55d6-36fb-4f5b-a0a3-f22eb80c80ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:33:51.772129 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 4/7] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:33:51 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 26 00:33:52.041418 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97197954-c9cc-4d4e-b51f-4573c70fea35 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:33:52.053356 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97197954-c9cc-4d4e-b51f-4573c70fea35 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:33:52.057463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c708e5-6895-4ce2-8804-c3a0040d7ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:33:52.058064 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c708e5-6895-4ce2-8804-c3a0040d7ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:52.058340 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c708e5-6895-4ce2-8804-c3a0040d7ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:52.058610 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c708e5-6895-4ce2-8804-c3a0040d7ab0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:52.066106 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c708e5-6895-4ce2-8804-c3a0040d7ab0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:33:52.067858 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c708e5-6895-4ce2-8804-c3a0040d7ab0 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:33:52.076391 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 4/8] 10.222.0.59 () {58 vars in 1324 bytes} [Fri Aug 26 00:33:52 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 00:33:52.082239 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c55d6-36fb-4f5b-a0a3-f22eb80c80ae None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:33:52.091246 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c55d6-36fb-4f5b-a0a3-f22eb80c80ae None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:33:52.095552 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7e05e-ca93-4e46-ba09-dd1cd5b14d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:33:52.096179 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7e05e-ca93-4e46-ba09-dd1cd5b14d3b admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:52.096466 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7e05e-ca93-4e46-ba09-dd1cd5b14d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:52.096769 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7e05e-ca93-4e46-ba09-dd1cd5b14d3b admin admin] PATH_INFO: `/v3/users/795669ccac0f433b900784c76b5b0142` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:52.098711 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f7e05e-ca93-4e46-ba09-dd1cd5b14d3b admin admin] RBAC: Authorizing `identity:get_user(user_id=795669ccac0f433b900784c76b5b0142)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:33:52.100565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f7e05e-ca93-4e46-ba09-dd1cd5b14d3b admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:33:52.102088 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 5/9] 10.222.0.59 () {56 vars in 1198 bytes} [Fri Aug 26 00:33:52 2022] GET /identity/v3/users/795669ccac0f433b900784c76b5b0142 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:33:54.053538 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7cb0b-ce4a-4612-9505-e33f22455265 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:54.054074 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7cb0b-ce4a-4612-9505-e33f22455265 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:54.054074 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7cb0b-ce4a-4612-9505-e33f22455265 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:54.055426 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 5/10] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:33:54.060618 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19a45b-33bb-4d41-9207-6d63dc961ac6 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:54.061011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19a45b-33bb-4d41-9207-6d63dc961ac6 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:54.061173 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19a45b-33bb-4d41-9207-6d63dc961ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:54.078580 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-5c19a45b-33bb-4d41-9207-6d63dc961ac6 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:33:54.079745 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-5c19a45b-33bb-4d41-9207-6d63dc961ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:33:54.119526 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d21ef6-29f8-4364-83fe-ab35a817440c None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:54.119875 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d21ef6-29f8-4364-83fe-ab35a817440c None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:54.120162 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d21ef6-29f8-4364-83fe-ab35a817440c None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:54.121599 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 6/11] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:33:54.127603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24634e-df54-4ab1-9169-c82afb8ab805 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:54.127894 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24634e-df54-4ab1-9169-c82afb8ab805 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:54.128172 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24634e-df54-4ab1-9169-c82afb8ab805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:54.158944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 6/12] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:33:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:33:54.170047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG passlib.handlers.bcrypt [None req-3e24634e-df54-4ab1-9169-c82afb8ab805 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=88254) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 26 00:33:54.170475 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG passlib.handlers.bcrypt [None req-3e24634e-df54-4ab1-9169-c82afb8ab805 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88254) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 26 00:33:54.215763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-3e24634e-df54-4ab1-9169-c82afb8ab805 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:33:54.217406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-3e24634e-df54-4ab1-9169-c82afb8ab805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:33:54.315195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 7/13] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:33:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:33:54.426854 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19a45b-33bb-4d41-9207-6d63dc961ac6 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:33:54.451248 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:33:54.529551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19a45b-33bb-4d41-9207-6d63dc961ac6 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:33:54.561692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc97fd1-6089-492b-b353-354f30f0c103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:33:54.562313 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc97fd1-6089-492b-b353-354f30f0c103 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:54.562591 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc97fd1-6089-492b-b353-354f30f0c103 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:54.562866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc97fd1-6089-492b-b353-354f30f0c103 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:54.574065 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc97fd1-6089-492b-b353-354f30f0c103 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:33:54.580654 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24634e-df54-4ab1-9169-c82afb8ab805 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:33:54.582674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-5cc97fd1-6089-492b-b353-354f30f0c103 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:33:54.582674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:33:54.582674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:33:54.582674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:33:54.582674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:33:54.582674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:33:54.582674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:33:54.582674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:33:54.582674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:33:54.582674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:33:54.582674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:33:54.582674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:33:54.582674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:33:54.582674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:33:54.582674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:33:54.582674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:33:54.583779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:33:54.583779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:33:54.583779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:33:54.583779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:33:54.583779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:33:54.583779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:33:54.583779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:33:54.583779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:33:54.583779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:33:54.583779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:33:54.583779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:33:54.583779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:33:54.583779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:33:54.583779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:33:54.583779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:33:54.583779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:33:54.584899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:33:54.584899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:33:54.584899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:33:54.584899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:33:54.584899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:33:54.584899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:33:54.584899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:33:54.584899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:33:54.584899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:33:54.584899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:33:54.584899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:33:54.584899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:33:54.584899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 7/14] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:33:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:33:54.589235 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc97fd1-6089-492b-b353-354f30f0c103 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:33:54.669324 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:33:54.687262 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24634e-df54-4ab1-9169-c82afb8ab805 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:33:54.695819 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc97fd1-6089-492b-b353-354f30f0c103 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:33:54.695956 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:33:54.720954 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c7a986-cf1a-47a6-8518-caaa3e7c52ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:33:54.721566 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c7a986-cf1a-47a6-8518-caaa3e7c52ff None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:54.722363 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c7a986-cf1a-47a6-8518-caaa3e7c52ff None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:54.722639 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c7a986-cf1a-47a6-8518-caaa3e7c52ff None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:54.723302 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c7a986-cf1a-47a6-8518-caaa3e7c52ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:33:54.723658 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c7a986-cf1a-47a6-8518-caaa3e7c52ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:33:54.725085 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c7a986-cf1a-47a6-8518-caaa3e7c52ff None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:33:54.725792 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877322f-6b59-4d6c-a214-21220479b47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:33:54.726716 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877322f-6b59-4d6c-a214-21220479b47d None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:54.727003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877322f-6b59-4d6c-a214-21220479b47d None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:54.727277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877322f-6b59-4d6c-a214-21220479b47d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:54.736544 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 8/15] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:33:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:33:54.742587 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c7a986-cf1a-47a6-8518-caaa3e7c52ff None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:33:54.794664 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a877322f-6b59-4d6c-a214-21220479b47d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:33:54.796297 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a877322f-6b59-4d6c-a214-21220479b47d None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:33:54.838194 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c7a986-cf1a-47a6-8518-caaa3e7c52ff None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:33:54.872234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b3598-50e6-4fb3-83c0-6314ed38deed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:33:54.872885 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b3598-50e6-4fb3-83c0-6314ed38deed None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:54.873161 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b3598-50e6-4fb3-83c0-6314ed38deed None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:54.873433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b3598-50e6-4fb3-83c0-6314ed38deed None admin] PATH_INFO: `/v3/users/795669ccac0f433b900784c76b5b0142` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:54.874551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24b3598-50e6-4fb3-83c0-6314ed38deed None admin] RBAC: Authorizing `identity:get_user(user_id=795669ccac0f433b900784c76b5b0142)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:33:54.875985 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24b3598-50e6-4fb3-83c0-6314ed38deed None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:33:54.877542 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 9/16] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:33:54 2022] GET /identity/v3/users/795669ccac0f433b900784c76b5b0142 => generated 247 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:33:54.883047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b3598-50e6-4fb3-83c0-6314ed38deed None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:33:54.894201 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 8/17] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:33:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:33:54.897000 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:33:54.899523 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877322f-6b59-4d6c-a214-21220479b47d None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:33:54.902227 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:33:54.981060 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b3598-50e6-4fb3-83c0-6314ed38deed None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:33:54.998549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877322f-6b59-4d6c-a214-21220479b47d None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:33:55.014854 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967eabb-a6ff-4c64-8263-9680aa7e8a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:33:55.015495 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967eabb-a6ff-4c64-8263-9680aa7e8a68 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:55.015773 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967eabb-a6ff-4c64-8263-9680aa7e8a68 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:55.016046 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967eabb-a6ff-4c64-8263-9680aa7e8a68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:55.017177 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9967eabb-a6ff-4c64-8263-9680aa7e8a68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:33:55.019505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9967eabb-a6ff-4c64-8263-9680aa7e8a68 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:33:55.021095 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 10/18] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:33:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:33:55.026883 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967eabb-a6ff-4c64-8263-9680aa7e8a68 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:33:55.032302 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a159f3-62a4-4550-a13d-9a7ba2baffa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:33:55.033274 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a159f3-62a4-4550-a13d-9a7ba2baffa8 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:55.033558 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a159f3-62a4-4550-a13d-9a7ba2baffa8 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:55.033831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a159f3-62a4-4550-a13d-9a7ba2baffa8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:55.053441 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a159f3-62a4-4550-a13d-9a7ba2baffa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:33:55.054902 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a159f3-62a4-4550-a13d-9a7ba2baffa8 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:33:55.072621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-83a159f3-62a4-4550-a13d-9a7ba2baffa8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:33:55.072621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:33:55.072621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:33:55.072621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:33:55.072621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:33:55.072621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:33:55.072621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:33:55.072621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:33:55.072621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:33:55.072621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:33:55.072621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:33:55.072621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:33:55.072621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:33:55.072621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:33:55.072621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:33:55.072621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:33:55.073667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:33:55.073667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:33:55.073667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:33:55.073667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:33:55.073667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:33:55.073667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:33:55.073667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:33:55.073667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:33:55.073667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:33:55.073667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:33:55.073667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:33:55.073667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:33:55.073667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:33:55.073667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:33:55.073667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:33:55.073667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:33:55.073667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:33:55.074674 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:33:55.074674 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:33:55.074674 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:33:55.074674 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 9/19] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:33:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:33:55.080707 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a159f3-62a4-4550-a13d-9a7ba2baffa8 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:33:55.106661 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:33:55.125303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967eabb-a6ff-4c64-8263-9680aa7e8a68 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:33:55.137235 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:33:55.159995 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce55b7-e886-4ac2-873f-d4f19f0a1d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:33:55.160624 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce55b7-e886-4ac2-873f-d4f19f0a1d11 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:55.160911 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce55b7-e886-4ac2-873f-d4f19f0a1d11 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:55.161187 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce55b7-e886-4ac2-873f-d4f19f0a1d11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:55.161908 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce55b7-e886-4ac2-873f-d4f19f0a1d11 None admin] RBAC: Adding query filter params (role.id=1a33e3c8cb8347feb9498595886cc621, scope.domain.id=default, user.id=795669ccac0f433b900784c76b5b0142) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:33:55.162216 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce55b7-e886-4ac2-873f-d4f19f0a1d11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:33:55.163668 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce55b7-e886-4ac2-873f-d4f19f0a1d11 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:33:55.170406 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 11/20] 10.222.0.59 () {56 vars in 1322 bytes} [Fri Aug 26 00:33:55 2022] GET /identity/v3/role_assignments?user.id=795669ccac0f433b900784c76b5b0142&scope.domain.id=default&role.id=1a33e3c8cb8347feb9498595886cc621 => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:33:55.181630 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a159f3-62a4-4550-a13d-9a7ba2baffa8 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:33:55.218603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6016bb14-9a2e-4189-878c-6c7ee26e50cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:33:55.219241 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016bb14-9a2e-4189-878c-6c7ee26e50cd None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:55.219522 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016bb14-9a2e-4189-878c-6c7ee26e50cd None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:55.219796 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016bb14-9a2e-4189-878c-6c7ee26e50cd None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:55.220459 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6016bb14-9a2e-4189-878c-6c7ee26e50cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:33:55.220792 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6016bb14-9a2e-4189-878c-6c7ee26e50cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:33:55.222987 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6016bb14-9a2e-4189-878c-6c7ee26e50cd None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:33:55.241887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 10/21] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:33:57.335464 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcba85a-2fb4-46da-bd43-9a538c570503 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:57.336013 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcba85a-2fb4-46da-bd43-9a538c570503 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:57.336013 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcba85a-2fb4-46da-bd43-9a538c570503 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:57.337573 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 12/22] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:33:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:33:57.353358 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565f5dc-0d7c-4e49-a173-d512fd423aa0 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:57.353839 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565f5dc-0d7c-4e49-a173-d512fd423aa0 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:57.353914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565f5dc-0d7c-4e49-a173-d512fd423aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:57.381483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-1565f5dc-0d7c-4e49-a173-d512fd423aa0 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:33:57.382707 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-1565f5dc-0d7c-4e49-a173-d512fd423aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:33:57.458414 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 11/23] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:33:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:33:57.767973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597f593-d9e6-4256-bfec-23a307a99677 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:57.769270 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597f593-d9e6-4256-bfec-23a307a99677 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:57.769767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597f593-d9e6-4256-bfec-23a307a99677 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:57.771411 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 13/24] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:33:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:33:57.778729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916a796-f38b-45c7-89f5-093f8c97236e None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:57.779344 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916a796-f38b-45c7-89f5-093f8c97236e None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:57.779842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916a796-f38b-45c7-89f5-093f8c97236e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:57.804899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597f593-d9e6-4256-bfec-23a307a99677 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:33:57.819118 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-b916a796-f38b-45c7-89f5-093f8c97236e None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:33:57.822083 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-b916a796-f38b-45c7-89f5-093f8c97236e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:33:57.896840 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:33:57.924029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597f593-d9e6-4256-bfec-23a307a99677 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:33:57.926242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 12/25] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:33:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:33:57.950303 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1435aa-5a90-47e6-833f-ba3931cb8da9 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:57.951015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1435aa-5a90-47e6-833f-ba3931cb8da9 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:57.951519 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1435aa-5a90-47e6-833f-ba3931cb8da9 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:57.953496 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 13/26] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:33:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:33:57.959282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b546cce9-e3db-4fd7-b0f5-c853901827d3 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:57.959809 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b546cce9-e3db-4fd7-b0f5-c853901827d3 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:57.960287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b546cce9-e3db-4fd7-b0f5-c853901827d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:57.975754 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5bf647-f147-4dd9-b1d4-450f8a5acc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:33:57.976404 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5bf647-f147-4dd9-b1d4-450f8a5acc67 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:57.976850 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5bf647-f147-4dd9-b1d4-450f8a5acc67 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:57.977129 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5bf647-f147-4dd9-b1d4-450f8a5acc67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:57.997899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-b546cce9-e3db-4fd7-b0f5-c853901827d3 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:33:57.999099 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-b546cce9-e3db-4fd7-b0f5-c853901827d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:33:58.001919 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5bf647-f147-4dd9-b1d4-450f8a5acc67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:33:58.007360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-7a5bf647-f147-4dd9-b1d4-450f8a5acc67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:33:58.007360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:33:58.007360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:33:58.007360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:33:58.007360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:33:58.007360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:33:58.007360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:33:58.007360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:33:58.007360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:33:58.007360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:33:58.007360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:33:58.007360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:33:58.007360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:33:58.007360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:33:58.007360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:33:58.007360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:33:58.013374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:33:58.013374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:33:58.013374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:33:58.013374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:33:58.013374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:33:58.013374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:33:58.013374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:33:58.013374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:33:58.013374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:33:58.013374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:33:58.013374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:33:58.013374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:33:58.013374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:33:58.013374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:33:58.013374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:33:58.013374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:33:58.014159 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:33:58.014159 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:33:58.014159 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:33:58.014159 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:33:58.014159 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:33:58.014159 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:33:58.014159 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:33:58.014159 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:33:58.014159 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:33:58.014159 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:33:58.014159 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:33:58.014159 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:33:58.014159 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 14/27] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:33:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:33:58.015902 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5bf647-f147-4dd9-b1d4-450f8a5acc67 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:33:58.024847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:33:58.091206 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 14/28] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:33:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:33:58.100121 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fb9a4-2659-4d80-b15a-94a4e2ea739a None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:58.100717 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fb9a4-2659-4d80-b15a-94a4e2ea739a None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:58.101228 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fb9a4-2659-4d80-b15a-94a4e2ea739a None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:58.102728 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 15/29] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:33:58.107843 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6380f347-f70d-4285-afb9-2f32d162b8f5 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:58.110847 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6380f347-f70d-4285-afb9-2f32d162b8f5 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:58.111339 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6380f347-f70d-4285-afb9-2f32d162b8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:58.152931 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-6380f347-f70d-4285-afb9-2f32d162b8f5 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:33:58.155707 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-6380f347-f70d-4285-afb9-2f32d162b8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:33:58.217981 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5bf647-f147-4dd9-b1d4-450f8a5acc67 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:33:58.279709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca1eb3-f80f-4139-b2f3-31b043bcfbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:33:58.280717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca1eb3-f80f-4139-b2f3-31b043bcfbdd None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:58.281323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca1eb3-f80f-4139-b2f3-31b043bcfbdd None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:58.281816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca1eb3-f80f-4139-b2f3-31b043bcfbdd None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:58.282862 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca1eb3-f80f-4139-b2f3-31b043bcfbdd None admin] RBAC: Adding query filter params (name=admin) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:33:58.283370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca1eb3-f80f-4139-b2f3-31b043bcfbdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:33:58.285924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca1eb3-f80f-4139-b2f3-31b043bcfbdd None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:33:58.299476 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 16/30] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:33:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:33:58.301967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 15/31] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:33:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:33:58.307520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6380f347-f70d-4285-afb9-2f32d162b8f5 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:33:58.344043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:33:58.410593 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca1eb3-f80f-4139-b2f3-31b043bcfbdd None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:33:58.426434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6380f347-f70d-4285-afb9-2f32d162b8f5 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:33:58.472119 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed62dd3-0e65-45cb-abb3-947a635fea1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:33:58.473215 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed62dd3-0e65-45cb-abb3-947a635fea1b None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:58.473849 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed62dd3-0e65-45cb-abb3-947a635fea1b None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:58.474317 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed62dd3-0e65-45cb-abb3-947a635fea1b None admin] PATH_INFO: `/v3/users/795669ccac0f433b900784c76b5b0142` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:58.475994 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed62dd3-0e65-45cb-abb3-947a635fea1b None admin] RBAC: Authorizing `identity:get_user(user_id=795669ccac0f433b900784c76b5b0142)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:33:58.477704 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed62dd3-0e65-45cb-abb3-947a635fea1b None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:33:58.479778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 17/32] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:33:58 2022] GET /identity/v3/users/795669ccac0f433b900784c76b5b0142 => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:33:58.481066 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed62dd3-0e65-45cb-abb3-947a635fea1b None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:33:58.552185 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:33:58.593975 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed62dd3-0e65-45cb-abb3-947a635fea1b None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:33:58.596807 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:33:58.632095 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca1eb3-f80f-4139-b2f3-31b043bcfbdd None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:33:58.632927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5395ce1-2041-450f-b626-8260f09b6fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:33:58.633872 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5395ce1-2041-450f-b626-8260f09b6fbb None admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:58.634351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5395ce1-2041-450f-b626-8260f09b6fbb None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:58.634838 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5395ce1-2041-450f-b626-8260f09b6fbb None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:58.635720 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5395ce1-2041-450f-b626-8260f09b6fbb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:33:58.637442 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5395ce1-2041-450f-b626-8260f09b6fbb None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:33:58.677146 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c42ce-d623-409d-bea5-4564c5affcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:33:58.678115 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c42ce-d623-409d-bea5-4564c5affcb1 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:58.678644 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c42ce-d623-409d-bea5-4564c5affcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:58.679169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c42ce-d623-409d-bea5-4564c5affcb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:58.691513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c42ce-d623-409d-bea5-4564c5affcb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:33:58.693304 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c42ce-d623-409d-bea5-4564c5affcb1 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:33:58.712446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-eb5c42ce-d623-409d-bea5-4564c5affcb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:33:58.712446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:33:58.712446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:33:58.712446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:33:58.712446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:33:58.712446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:33:58.712446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:33:58.712446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:33:58.712446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:33:58.712446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:33:58.712446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:33:58.712446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:33:58.712446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:33:58.712446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:33:58.712446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:33:58.712446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:33:58.713392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:33:58.713392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:33:58.713392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:33:58.713392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:33:58.713392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:33:58.713392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:33:58.713392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:33:58.713392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:33:58.713392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:33:58.713392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:33:58.713392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:33:58.713392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:33:58.713392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:33:58.713392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:33:58.713392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:33:58.713392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:33:58.713392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:33:58.714246 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:33:58.714246 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:33:58.714246 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:33:58.716191 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 16/33] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:33:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:33:58.717396 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c42ce-d623-409d-bea5-4564c5affcb1 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:33:58.833242 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c42ce-d623-409d-bea5-4564c5affcb1 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:33:58.850517 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:33:58.867299 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f36d04-1606-4d58-9259-9ba606f0476b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:33:58.867988 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f36d04-1606-4d58-9259-9ba606f0476b None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:58.868270 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f36d04-1606-4d58-9259-9ba606f0476b None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:58.868578 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f36d04-1606-4d58-9259-9ba606f0476b None admin] PATH_INFO: `/v3/domains/default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:58.869675 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f36d04-1606-4d58-9259-9ba606f0476b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:33:58.871169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f36d04-1606-4d58-9259-9ba606f0476b None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:33:58.872942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 17/34] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:33:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:33:58.874171 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f36d04-1606-4d58-9259-9ba606f0476b None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:33:58.899564 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 18/35] 10.222.0.59 () {60 vars in 1131 bytes} [Fri Aug 26 00:33:58 2022] POST /identity/v3/roles => generated 225 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:33:58.901568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5395ce1-2041-450f-b626-8260f09b6fbb None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:33:59.001409 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f36d04-1606-4d58-9259-9ba606f0476b None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:33:59.025939 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5395ce1-2041-450f-b626-8260f09b6fbb None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:33:59.039816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca5113-5016-4fda-a6f2-43196832ba25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:33:59.040444 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca5113-5016-4fda-a6f2-43196832ba25 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:59.040762 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca5113-5016-4fda-a6f2-43196832ba25 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:59.041040 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca5113-5016-4fda-a6f2-43196832ba25 None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:59.041756 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca5113-5016-4fda-a6f2-43196832ba25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:33:59.042068 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca5113-5016-4fda-a6f2-43196832ba25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:33:59.044362 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca5113-5016-4fda-a6f2-43196832ba25 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:33:59.066236 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 18/36] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:33:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:33:59.068918 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2877215-73fe-436e-886e-c85c77e88914 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:59.069420 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2877215-73fe-436e-886e-c85c77e88914 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:59.069896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2877215-73fe-436e-886e-c85c77e88914 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:59.071426 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22d5a8-2378-4b67-9fb6-19fa65b76e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:33:59.071928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 19/37] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:33:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:33:59.072159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22d5a8-2378-4b67-9fb6-19fa65b76e62 None admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:59.072412 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22d5a8-2378-4b67-9fb6-19fa65b76e62 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:59.072713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22d5a8-2378-4b67-9fb6-19fa65b76e62 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:59.073384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d22d5a8-2378-4b67-9fb6-19fa65b76e62 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:33:59.074916 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d22d5a8-2378-4b67-9fb6-19fa65b76e62 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:33:59.076652 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2877215-73fe-436e-886e-c85c77e88914 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:33:59.135177 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:33:59.162088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 19/38] 10.222.0.59 () {60 vars in 1131 bytes} [Fri Aug 26 00:33:58 2022] POST /identity/v3/roles => generated 219 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:33:59.163301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22d5a8-2378-4b67-9fb6-19fa65b76e62 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:33:59.197292 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2877215-73fe-436e-886e-c85c77e88914 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:33:59.228867 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:33:59.240020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb55dd-fcbf-4cd2-8379-d709f6742f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:33:59.240921 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb55dd-fcbf-4cd2-8379-d709f6742f9b None admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:59.241276 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb55dd-fcbf-4cd2-8379-d709f6742f9b None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:59.241626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb55dd-fcbf-4cd2-8379-d709f6742f9b None admin] PATH_INFO: `/v3/domains/default/users/795669ccac0f433b900784c76b5b0142/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:59.244430 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cb55dd-fcbf-4cd2-8379-d709f6742f9b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=795669ccac0f433b900784c76b5b0142, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:33:59.246385 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cb55dd-fcbf-4cd2-8379-d709f6742f9b None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:33:59.298736 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22d5a8-2378-4b67-9fb6-19fa65b76e62 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:33:59.338285 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd7297-076c-416e-9828-0bc705d407d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:33:59.338907 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd7297-076c-416e-9828-0bc705d407d5 None admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:59.339187 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd7297-076c-416e-9828-0bc705d407d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:59.339462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd7297-076c-416e-9828-0bc705d407d5 None admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:59.340138 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd7297-076c-416e-9828-0bc705d407d5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:33:59.341739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd7297-076c-416e-9828-0bc705d407d5 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:33:59.443039 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 20/39] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Aug 26 00:33:59 2022] POST /identity/v3/projects => generated 301 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:33:59.444591 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffaa566-e4e0-42e0-93c8-e5e0f81b5086 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:59.444881 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffaa566-e4e0-42e0-93c8-e5e0f81b5086 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:59.445156 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffaa566-e4e0-42e0-93c8-e5e0f81b5086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:59.465486 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-cffaa566-e4e0-42e0-93c8-e5e0f81b5086 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:33:59.467212 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-cffaa566-e4e0-42e0-93c8-e5e0f81b5086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:33:59.556518 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 21/40] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:33:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:33:59.556518 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca961076-facd-411b-b950-182e9dfa00b0 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:59.556518 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca961076-facd-411b-b950-182e9dfa00b0 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:59.556518 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca961076-facd-411b-b950-182e9dfa00b0 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:59.556518 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 22/41] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:33:59.561288 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0b5d8-b8db-454a-9879-0396385a4ae9 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:59.561579 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0b5d8-b8db-454a-9879-0396385a4ae9 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:59.561840 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0b5d8-b8db-454a-9879-0396385a4ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:59.575460 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:33:59.576952 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 20/42] 10.222.0.59 () {58 vars in 1361 bytes} [Fri Aug 26 00:33:59 2022] PUT /identity/v3/domains/default/users/795669ccac0f433b900784c76b5b0142/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:33:59.583861 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-0ed0b5d8-b8db-454a-9879-0396385a4ae9 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:33:59.585643 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-0ed0b5d8-b8db-454a-9879-0396385a4ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:33:59.677167 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 23/43] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:33:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:33:59.833326 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb55dd-fcbf-4cd2-8379-d709f6742f9b None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:33:59.937000 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb55dd-fcbf-4cd2-8379-d709f6742f9b None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:33:59.945021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:33:59.952273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed0b5d8-b8db-454a-9879-0396385a4ae9 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:33:59.973506 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d3210-2c6b-4d6b-ab81-c6255b98b0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:33:59.974408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d3210-2c6b-4d6b-ab81-c6255b98b0db None admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:33:59.974692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d3210-2c6b-4d6b-ab81-c6255b98b0db None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:33:59.974970 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d3210-2c6b-4d6b-ab81-c6255b98b0db None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:33:59.975716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d3210-2c6b-4d6b-ab81-c6255b98b0db None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:33:59.977264 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d3210-2c6b-4d6b-ab81-c6255b98b0db None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:00.055833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed0b5d8-b8db-454a-9879-0396385a4ae9 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:00.071748 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 21/44] 10.222.0.59 () {60 vars in 1131 bytes} [Fri Aug 26 00:33:59 2022] POST /identity/v3/roles => generated 223 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:34:00.093325 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525748d4-9124-4ca7-958f-17ee21768486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:00.093916 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525748d4-9124-4ca7-958f-17ee21768486 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:00.094194 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525748d4-9124-4ca7-958f-17ee21768486 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:00.094463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525748d4-9124-4ca7-958f-17ee21768486 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:00.095762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525748d4-9124-4ca7-958f-17ee21768486 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:00.097327 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525748d4-9124-4ca7-958f-17ee21768486 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:00.099354 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 24/45] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:33:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:00.104880 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d3210-2c6b-4d6b-ab81-c6255b98b0db None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:00.220222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d3210-2c6b-4d6b-ab81-c6255b98b0db None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:00.257321 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5caac7-fea5-4890-bef1-5a3c37867926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:00.257948 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5caac7-fea5-4890-bef1-5a3c37867926 None admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:00.258309 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5caac7-fea5-4890-bef1-5a3c37867926 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:00.258522 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5caac7-fea5-4890-bef1-5a3c37867926 None admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:00.259186 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5caac7-fea5-4890-bef1-5a3c37867926 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:00.260682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5caac7-fea5-4890-bef1-5a3c37867926 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:00.359801 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 22/46] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Aug 26 00:34:00 2022] POST /identity/v3/projects => generated 312 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:34:01.492021 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768bbc9-4f72-46c1-bf48-a245d262914f None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:01.492552 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768bbc9-4f72-46c1-bf48-a245d262914f None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:01.492639 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768bbc9-4f72-46c1-bf48-a245d262914f None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:01.493934 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 25/47] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:01.499095 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd0faa-6abe-4bba-b57c-fefb4c728c46 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:01.499506 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd0faa-6abe-4bba-b57c-fefb4c728c46 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:01.499655 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd0faa-6abe-4bba-b57c-fefb4c728c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:01.517536 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-b5bd0faa-6abe-4bba-b57c-fefb4c728c46 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:01.519058 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-b5bd0faa-6abe-4bba-b57c-fefb4c728c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:01.531956 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:01.583325 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 23/48] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:01.846180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6768bbc9-4f72-46c1-bf48-a245d262914f None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:01.850737 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:01.948585 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6768bbc9-4f72-46c1-bf48-a245d262914f None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:01.982172 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67c943-b521-4d5e-a645-47afd7e64aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:01.982802 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67c943-b521-4d5e-a645-47afd7e64aa4 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:01.983081 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67c943-b521-4d5e-a645-47afd7e64aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:01.983355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67c943-b521-4d5e-a645-47afd7e64aa4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:01.994130 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb67c943-b521-4d5e-a645-47afd7e64aa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:02.104209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-eb67c943-b521-4d5e-a645-47afd7e64aa4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:34:02.104209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:02.104209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:02.104209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:02.104209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:02.104209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:02.104209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:02.104209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:02.104209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:02.104209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:02.104209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:02.104209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:02.104209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:02.104209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:02.104209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:02.104209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:02.105159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:02.105159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:02.105159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:02.105159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:02.105159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:02.105159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:02.105159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:02.105159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:02.105159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:02.105159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:02.105159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:02.105159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:02.105159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:02.105159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:02.105159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:02.105159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:02.106009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:02.106009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:02.106009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:02.106009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:02.106009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:02.106009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:02.106009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:02.106009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:02.106009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:02.106009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:02.106009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:34:02.106009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:34:02.106009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 26/49] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:34:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:02.111084 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd0faa-6abe-4bba-b57c-fefb4c728c46 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:02.208411 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd0faa-6abe-4bba-b57c-fefb4c728c46 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:02.243458 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c325a-ccec-45b2-9c4f-48c3e9b16071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:02.244080 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c325a-ccec-45b2-9c4f-48c3e9b16071 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:02.244365 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c325a-ccec-45b2-9c4f-48c3e9b16071 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:02.244672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c325a-ccec-45b2-9c4f-48c3e9b16071 None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:02.245351 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c325a-ccec-45b2-9c4f-48c3e9b16071 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:02.245730 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c325a-ccec-45b2-9c4f-48c3e9b16071 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:02.247157 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc5c20-040b-41c8-9a20-5a85c482f436 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:02.247291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c325a-ccec-45b2-9c4f-48c3e9b16071 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:02.247439 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc5c20-040b-41c8-9a20-5a85c482f436 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:02.247713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc5c20-040b-41c8-9a20-5a85c482f436 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:02.249085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 27/50] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:02.254376 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc887e2c-1905-4671-8279-d5b35adcc34e None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:02.254664 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc887e2c-1905-4671-8279-d5b35adcc34e None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:02.254940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc887e2c-1905-4671-8279-d5b35adcc34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:02.256118 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 24/51] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:02.261379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c325a-ccec-45b2-9c4f-48c3e9b16071 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:02.273138 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-cc887e2c-1905-4671-8279-d5b35adcc34e None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:02.274036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:02.274346 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-cc887e2c-1905-4671-8279-d5b35adcc34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:02.333581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:02.341702 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 28/52] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:34:02.359811 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c325a-ccec-45b2-9c4f-48c3e9b16071 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:02.392111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e131a95a-f1f0-4c4d-bea3-de987f450661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:02.392762 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131a95a-f1f0-4c4d-bea3-de987f450661 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:02.393055 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131a95a-f1f0-4c4d-bea3-de987f450661 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:02.393874 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131a95a-f1f0-4c4d-bea3-de987f450661 None admin] PATH_INFO: `/v3/users/795669ccac0f433b900784c76b5b0142` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:02.395017 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e131a95a-f1f0-4c4d-bea3-de987f450661 None admin] RBAC: Authorizing `identity:get_user(user_id=795669ccac0f433b900784c76b5b0142)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:02.396465 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e131a95a-f1f0-4c4d-bea3-de987f450661 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:02.398059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 25/53] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:02 2022] GET /identity/v3/users/795669ccac0f433b900784c76b5b0142 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:02.403880 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc887e2c-1905-4671-8279-d5b35adcc34e None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:02.499652 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc887e2c-1905-4671-8279-d5b35adcc34e None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:02.532444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c73de6-0a81-468c-bfc5-fe554dcc029c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:02.533102 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c73de6-0a81-468c-bfc5-fe554dcc029c None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:02.533385 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c73de6-0a81-468c-bfc5-fe554dcc029c None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:02.533660 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c73de6-0a81-468c-bfc5-fe554dcc029c None admin] PATH_INFO: `/v3/domains/default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:02.534832 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c73de6-0a81-468c-bfc5-fe554dcc029c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:02.536283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c73de6-0a81-468c-bfc5-fe554dcc029c None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:02.537863 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 29/54] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:34:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:02.543598 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e131a95a-f1f0-4c4d-bea3-de987f450661 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:02.608108 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c73de6-0a81-468c-bfc5-fe554dcc029c None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:02.619987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:02.640985 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e131a95a-f1f0-4c4d-bea3-de987f450661 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:02.668918 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:02.674231 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130316e-843a-4779-990a-3514319e009e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:02.675441 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130316e-843a-4779-990a-3514319e009e None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:02.675731 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130316e-843a-4779-990a-3514319e009e None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:02.676011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130316e-843a-4779-990a-3514319e009e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:02.676770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130316e-843a-4779-990a-3514319e009e None admin] RBAC: Adding query filter params (role.id=1a33e3c8cb8347feb9498595886cc621, scope.domain.id=default, user.id=795669ccac0f433b900784c76b5b0142) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:02.677083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130316e-843a-4779-990a-3514319e009e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:02.678529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130316e-843a-4779-990a-3514319e009e None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:02.684202 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 26/55] 10.222.0.59 () {56 vars in 1322 bytes} [Fri Aug 26 00:34:02 2022] GET /identity/v3/role_assignments?user.id=795669ccac0f433b900784c76b5b0142&scope.domain.id=default&role.id=1a33e3c8cb8347feb9498595886cc621 => generated 530 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:02.710272 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c73de6-0a81-468c-bfc5-fe554dcc029c None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:02.747218 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0dbe75-ce54-4301-a381-decc2fb384e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:02.747881 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0dbe75-ce54-4301-a381-decc2fb384e8 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:02.748169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0dbe75-ce54-4301-a381-decc2fb384e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:02.748449 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0dbe75-ce54-4301-a381-decc2fb384e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:02.749653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0dbe75-ce54-4301-a381-decc2fb384e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:02.751139 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0dbe75-ce54-4301-a381-decc2fb384e8 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:02.753320 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 30/56] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:34:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:02.759078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130316e-843a-4779-990a-3514319e009e None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:02.861789 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130316e-843a-4779-990a-3514319e009e None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:02.895368 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61ef337-1671-4030-b187-7bde7fa0d370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:02.895919 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ef337-1671-4030-b187-7bde7fa0d370 None admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:02.896206 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ef337-1671-4030-b187-7bde7fa0d370 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:02.896518 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ef337-1671-4030-b187-7bde7fa0d370 None admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:02.897172 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61ef337-1671-4030-b187-7bde7fa0d370 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:02.898633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61ef337-1671-4030-b187-7bde7fa0d370 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:02.913730 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:02.982265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 27/57] 10.222.0.59 () {60 vars in 1140 bytes} [Fri Aug 26 00:34:02 2022] POST /identity/v3/projects => generated 298 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:34:04.869467 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f64269-9a48-4bf7-89b4-34f5fe905f6c None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:04.869968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f64269-9a48-4bf7-89b4-34f5fe905f6c None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:04.870051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f64269-9a48-4bf7-89b4-34f5fe905f6c None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:04.871345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 31/58] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:04.876630 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18dbdb-ee9f-4257-836b-6064c3f05070 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:04.877051 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18dbdb-ee9f-4257-836b-6064c3f05070 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:04.877180 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18dbdb-ee9f-4257-836b-6064c3f05070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:04.895059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-9e18dbdb-ee9f-4257-836b-6064c3f05070 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:04.896234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-9e18dbdb-ee9f-4257-836b-6064c3f05070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:04.959956 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 28/59] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:05.233237 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f64269-9a48-4bf7-89b4-34f5fe905f6c None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:05.343051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f64269-9a48-4bf7-89b4-34f5fe905f6c None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:05.370505 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:05.375925 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e065bbe-7dc8-4fbf-a62b-26d9cce8e5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:05.376580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e065bbe-7dc8-4fbf-a62b-26d9cce8e5bb None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:05.376860 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e065bbe-7dc8-4fbf-a62b-26d9cce8e5bb None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:05.377132 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e065bbe-7dc8-4fbf-a62b-26d9cce8e5bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:05.378732 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e065bbe-7dc8-4fbf-a62b-26d9cce8e5bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:05.380174 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e065bbe-7dc8-4fbf-a62b-26d9cce8e5bb None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:05.381742 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 32/60] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:34:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:05.387304 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18dbdb-ee9f-4257-836b-6064c3f05070 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:05.483124 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18dbdb-ee9f-4257-836b-6064c3f05070 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:05.506403 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:05.516509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b543b4-fa79-4689-9964-239db38c0411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:05.517116 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b543b4-fa79-4689-9964-239db38c0411 None admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:05.522006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b543b4-fa79-4689-9964-239db38c0411 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:05.522006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b543b4-fa79-4689-9964-239db38c0411 None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:05.522006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b543b4-fa79-4689-9964-239db38c0411 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:05.522006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b543b4-fa79-4689-9964-239db38c0411 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:05.650410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 29/61] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Aug 26 00:34:05 2022] POST /identity/v3/users => generated 275 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:34:08.501935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd5bcb-b056-4ef1-a24b-f35a448e98fa None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:08.502467 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd5bcb-b056-4ef1-a24b-f35a448e98fa None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:08.502567 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd5bcb-b056-4ef1-a24b-f35a448e98fa None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:08.503971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 33/62] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:08.509360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2c6b5-1993-4e43-b016-762213ea43f6 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:08.510189 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2c6b5-1993-4e43-b016-762213ea43f6 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:08.510670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2c6b5-1993-4e43-b016-762213ea43f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:08.545546 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-48a2c6b5-1993-4e43-b016-762213ea43f6 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:08.546822 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-48a2c6b5-1993-4e43-b016-762213ea43f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:08.638964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 30/63] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:34:08.765354 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4bb3a-2dc5-44d6-8e39-e5fbc030ab19 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:08.765544 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4bb3a-2dc5-44d6-8e39-e5fbc030ab19 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:08.765973 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4bb3a-2dc5-44d6-8e39-e5fbc030ab19 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:08.767461 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 34/64] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:08.777950 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895bcfe2-27ed-45e3-9ee1-e4faf473142e None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:08.778243 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895bcfe2-27ed-45e3-9ee1-e4faf473142e None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:08.778792 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895bcfe2-27ed-45e3-9ee1-e4faf473142e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:08.812602 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-895bcfe2-27ed-45e3-9ee1-e4faf473142e None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:08.813793 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-895bcfe2-27ed-45e3-9ee1-e4faf473142e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:08.913830 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 31/65] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:34:08.919043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cee6cf-c84e-4ae8-b8b4-cdad49c560de None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:08.919043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cee6cf-c84e-4ae8-b8b4-cdad49c560de None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:08.919043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cee6cf-c84e-4ae8-b8b4-cdad49c560de None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:08.919043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 35/66] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:08.934345 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026facfd-3d05-4fe7-b5f0-0b8f7e3d9901 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:08.934638 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026facfd-3d05-4fe7-b5f0-0b8f7e3d9901 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:08.934922 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026facfd-3d05-4fe7-b5f0-0b8f7e3d9901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:08.957537 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-026facfd-3d05-4fe7-b5f0-0b8f7e3d9901 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:08.958731 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-026facfd-3d05-4fe7-b5f0-0b8f7e3d9901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:08.977082 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cee6cf-c84e-4ae8-b8b4-cdad49c560de None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:09.020224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:09.071086 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 32/67] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:09.136520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cee6cf-c84e-4ae8-b8b4-cdad49c560de None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:09.185972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c8d90-3922-41c4-92b8-b12b50ee5af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:09.187036 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c8d90-3922-41c4-92b8-b12b50ee5af5 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:09.187511 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c8d90-3922-41c4-92b8-b12b50ee5af5 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:09.187978 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c8d90-3922-41c4-92b8-b12b50ee5af5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:09.199413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c8d90-3922-41c4-92b8-b12b50ee5af5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:09.204618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-402c8d90-3922-41c4-92b8-b12b50ee5af5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:09.204618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:09.204618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:09.204618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:09.204618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:09.204618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:09.204618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:09.204618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:09.204618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:09.204618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:09.204618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:09.204618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:09.204618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:09.204618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:09.204618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:09.204618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:09.205527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:09.205527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:09.205527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:09.205527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:09.205527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:09.205527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:09.205527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:09.205527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:09.205527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:09.205527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:09.205527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:09.205527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:09.205527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:09.205527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:09.205527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:09.205527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:09.206307 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:09.206307 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:09.206307 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:09.206307 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:09.206307 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:09.206307 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:09.206307 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:09.206307 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:09.206307 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:09.206307 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:09.206307 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:09.206307 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:34:09.208606 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 36/68] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:34:09.217578 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026facfd-3d05-4fe7-b5f0-0b8f7e3d9901 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:09.278701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c8d90-3922-41c4-92b8-b12b50ee5af5 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:09.355855 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026facfd-3d05-4fe7-b5f0-0b8f7e3d9901 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:09.368201 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:09.391774 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9e106-0234-429d-b669-8ecd460e1e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:09.392391 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9e106-0234-429d-b669-8ecd460e1e3c None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:09.392707 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9e106-0234-429d-b669-8ecd460e1e3c None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:09.393000 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9e106-0234-429d-b669-8ecd460e1e3c None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:09.393667 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9e106-0234-429d-b669-8ecd460e1e3c None admin] RBAC: Adding query filter params (name=member) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:09.393969 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9e106-0234-429d-b669-8ecd460e1e3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:09.395370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9e106-0234-429d-b669-8ecd460e1e3c None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:09.404713 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 33/69] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:34:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:09.409840 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f5f0f-62fd-4d5d-9002-560097a3b084 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:09.410128 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f5f0f-62fd-4d5d-9002-560097a3b084 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:09.410407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f5f0f-62fd-4d5d-9002-560097a3b084 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:09.411655 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 34/70] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:09.412359 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f5f0f-62fd-4d5d-9002-560097a3b084 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:09.425347 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c8d90-3922-41c4-92b8-b12b50ee5af5 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:09.448063 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:09.464591 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb634372-64ab-430f-97b8-7a117f5937ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:09.465329 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb634372-64ab-430f-97b8-7a117f5937ee None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:09.465648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb634372-64ab-430f-97b8-7a117f5937ee None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:09.465866 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb634372-64ab-430f-97b8-7a117f5937ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:09.476381 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb634372-64ab-430f-97b8-7a117f5937ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:09.481565 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-bb634372-64ab-430f-97b8-7a117f5937ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:34:09.481565 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:09.481565 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:09.481565 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:09.481565 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:09.481565 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:09.481565 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:09.481565 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:09.481565 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:09.481565 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:09.481565 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:09.481565 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:09.481565 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:09.481565 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:09.481565 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:09.481565 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:09.482598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:09.482598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:09.482598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:09.482598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:09.482598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:09.482598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:09.482598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:09.482598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:09.482598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:09.482598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:09.482598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:09.482598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:09.482598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:09.482598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:09.482598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:09.482598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:09.483791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:09.483791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:09.483791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:09.483791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:09.483791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:09.483791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:09.483791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:09.483791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:09.483791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:09.483791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:09.483791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:34:09.483791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:34:09.483791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 37/71] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:34:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:34:09.483791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb634372-64ab-430f-97b8-7a117f5937ee None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:09.526838 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f5f0f-62fd-4d5d-9002-560097a3b084 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:09.559512 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71315c2e-83cb-4d9d-9114-65cc96188ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:09.560142 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71315c2e-83cb-4d9d-9114-65cc96188ca0 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:09.560424 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71315c2e-83cb-4d9d-9114-65cc96188ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:09.560724 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71315c2e-83cb-4d9d-9114-65cc96188ca0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:09.570903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71315c2e-83cb-4d9d-9114-65cc96188ca0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:09.575589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-71315c2e-83cb-4d9d-9114-65cc96188ca0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 00:34:09.575589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:09.575589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:09.575589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:09.575589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:09.575589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:09.575589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:09.575589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:09.575589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:09.575589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:09.575589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:09.575589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:09.575589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:09.575589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:09.575589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:09.575589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:09.576616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:09.576616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:09.576616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:09.576616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:09.576616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:09.576616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:09.576616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:09.576616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:09.576616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:09.576616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:09.576616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:09.576616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:09.576616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:09.576616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:09.576616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:09.576616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:09.577398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:09.577398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:09.577398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:09.577398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:09.577398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:09.577398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:09.577398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:09.577398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:09.577398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:09.577398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:09.577398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 00:34:09.577398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:34:09.577398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 35/72] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Aug 26 00:34:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:09.578983 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc58179-26fc-47ae-b479-fe7b14df1956 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:09.579270 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc58179-26fc-47ae-b479-fe7b14df1956 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:09.579547 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc58179-26fc-47ae-b479-fe7b14df1956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:09.586725 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb634372-64ab-430f-97b8-7a117f5937ee None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:09.597137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-7dc58179-26fc-47ae-b479-fe7b14df1956 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:09.598378 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-7dc58179-26fc-47ae-b479-fe7b14df1956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:09.604155 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:09.621419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c686fb9-61ec-452c-aee1-3110e57a4f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:09.622062 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c686fb9-61ec-452c-aee1-3110e57a4f0e None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:09.622592 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c686fb9-61ec-452c-aee1-3110e57a4f0e None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:09.622592 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c686fb9-61ec-452c-aee1-3110e57a4f0e None admin] PATH_INFO: `/v3/users/b92a689fcc4842b3aa1d3a5c3f5e47b4` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:09.637369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c686fb9-61ec-452c-aee1-3110e57a4f0e None admin] RBAC: Authorizing `identity:get_user(user_id=b92a689fcc4842b3aa1d3a5c3f5e47b4)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:09.638837 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c686fb9-61ec-452c-aee1-3110e57a4f0e None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:09.640480 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 38/73] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:09 2022] GET /identity/v3/users/b92a689fcc4842b3aa1d3a5c3f5e47b4 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:09.642048 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c686fb9-61ec-452c-aee1-3110e57a4f0e None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:09.664162 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 36/74] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:09.667936 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf1af5-90bb-4447-9839-35eb035f02ee None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:09.668268 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf1af5-90bb-4447-9839-35eb035f02ee None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:09.668644 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf1af5-90bb-4447-9839-35eb035f02ee None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:09.670086 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 37/75] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:09.670780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf1af5-90bb-4447-9839-35eb035f02ee None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:09.699398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:09.753621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c686fb9-61ec-452c-aee1-3110e57a4f0e None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:09.778717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf1af5-90bb-4447-9839-35eb035f02ee None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:09.787288 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9923b23e-9321-4bd6-9767-e48fbad2383b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:09.787910 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923b23e-9321-4bd6-9767-e48fbad2383b None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:09.788198 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923b23e-9321-4bd6-9767-e48fbad2383b None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:09.788480 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923b23e-9321-4bd6-9767-e48fbad2383b None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:09.789175 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9923b23e-9321-4bd6-9767-e48fbad2383b None admin] RBAC: Adding query filter params (name=admin) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:09.789485 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9923b23e-9321-4bd6-9767-e48fbad2383b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:09.790981 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9923b23e-9321-4bd6-9767-e48fbad2383b None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:09.802965 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 39/76] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:09.808338 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9923b23e-9321-4bd6-9767-e48fbad2383b None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:09.814395 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff3a28-0c5c-4e1a-888c-df5f8c8398c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:09.815760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff3a28-0c5c-4e1a-888c-df5f8c8398c3 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:09.816217 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff3a28-0c5c-4e1a-888c-df5f8c8398c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:09.816692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff3a28-0c5c-4e1a-888c-df5f8c8398c3 None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:09.817551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff3a28-0c5c-4e1a-888c-df5f8c8398c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:09.818028 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff3a28-0c5c-4e1a-888c-df5f8c8398c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:09.819571 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff3a28-0c5c-4e1a-888c-df5f8c8398c3 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:09.829350 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 38/77] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:34:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:09.830834 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468a583-e9d1-4ca5-8677-7daa531d1ed5 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:09.831320 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468a583-e9d1-4ca5-8677-7daa531d1ed5 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:09.831769 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468a583-e9d1-4ca5-8677-7daa531d1ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:09.850073 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-e468a583-e9d1-4ca5-8677-7daa531d1ed5 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:09.851498 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-e468a583-e9d1-4ca5-8677-7daa531d1ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:09.853580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:09.915626 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9923b23e-9321-4bd6-9767-e48fbad2383b None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:09.919620 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 39/78] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:09.925008 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468a583-e9d1-4ca5-8677-7daa531d1ed5 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:09.940279 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:09.960583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b284ef-62ea-442c-9cec-61c877a792d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:09.961207 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b284ef-62ea-442c-9cec-61c877a792d5 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:09.961617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b284ef-62ea-442c-9cec-61c877a792d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:09.961911 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b284ef-62ea-442c-9cec-61c877a792d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:10.032590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b284ef-62ea-442c-9cec-61c877a792d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:10.034158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b284ef-62ea-442c-9cec-61c877a792d5 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:10.042932 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468a583-e9d1-4ca5-8677-7daa531d1ed5 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:10.075412 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:10.080355 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232395a-9a56-47f4-8e9c-b5dc4f2ce6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:10.081668 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232395a-9a56-47f4-8e9c-b5dc4f2ce6d1 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:10.081963 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232395a-9a56-47f4-8e9c-b5dc4f2ce6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:10.082302 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232395a-9a56-47f4-8e9c-b5dc4f2ce6d1 None admin] PATH_INFO: `/v3/users/795669ccac0f433b900784c76b5b0142` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:10.083483 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2232395a-9a56-47f4-8e9c-b5dc4f2ce6d1 None admin] RBAC: Authorizing `identity:get_user(user_id=795669ccac0f433b900784c76b5b0142)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:10.084977 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2232395a-9a56-47f4-8e9c-b5dc4f2ce6d1 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:10.086783 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 40/79] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:09 2022] GET /identity/v3/users/795669ccac0f433b900784c76b5b0142 => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:10.092248 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232395a-9a56-47f4-8e9c-b5dc4f2ce6d1 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:10.139399 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 40/80] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:34:10.140070 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b284ef-62ea-442c-9cec-61c877a792d5 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:10.182428 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:10.199975 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232395a-9a56-47f4-8e9c-b5dc4f2ce6d1 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:10.233479 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162ca26-0f02-450c-953e-444fd69132da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:10.234118 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162ca26-0f02-450c-953e-444fd69132da None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:10.234405 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162ca26-0f02-450c-953e-444fd69132da None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:10.234683 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162ca26-0f02-450c-953e-444fd69132da None admin] PATH_INFO: `/v3/users/b92a689fcc4842b3aa1d3a5c3f5e47b4` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:10.235814 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2162ca26-0f02-450c-953e-444fd69132da None admin] RBAC: Authorizing `identity:get_user(user_id=b92a689fcc4842b3aa1d3a5c3f5e47b4)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:10.237831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2162ca26-0f02-450c-953e-444fd69132da None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:10.239409 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 41/81] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:10 2022] GET /identity/v3/users/b92a689fcc4842b3aa1d3a5c3f5e47b4 => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:10.244183 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162ca26-0f02-450c-953e-444fd69132da None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:10.244284 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b284ef-62ea-442c-9cec-61c877a792d5 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:10.275156 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a471b79-0456-4881-bfb8-65a67bdfacd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:10.275780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a471b79-0456-4881-bfb8-65a67bdfacd1 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:10.276059 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a471b79-0456-4881-bfb8-65a67bdfacd1 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:10.276331 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a471b79-0456-4881-bfb8-65a67bdfacd1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:10.283249 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:10.286941 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a471b79-0456-4881-bfb8-65a67bdfacd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:10.291800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-4a471b79-0456-4881-bfb8-65a67bdfacd1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:10.291800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:10.291800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:10.291800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:10.291800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:10.291800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:10.291800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:10.291800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:10.291800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:10.291800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:10.291800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:10.291800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:10.291800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:10.291800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:10.291800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:10.291800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:10.297377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:10.297377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:10.297377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:10.297377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:10.297377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:10.297377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:10.297377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:10.297377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:10.297377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:10.297377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:10.297377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:10.297377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:10.297377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:10.297377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:10.297377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:10.297377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:10.298145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:10.298145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:10.298145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:10.298145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:10.298145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:10.298145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:10.298145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:10.298145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:10.298145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:10.298145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:10.298145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:10.298145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:34:10.298145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 41/82] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:10.298145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a471b79-0456-4881-bfb8-65a67bdfacd1 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:10.345944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162ca26-0f02-450c-953e-444fd69132da None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:10.388576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a781f3c1-49b8-4dff-be54-90aed01472df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:10.389618 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781f3c1-49b8-4dff-be54-90aed01472df None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:10.390118 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781f3c1-49b8-4dff-be54-90aed01472df None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:10.390610 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781f3c1-49b8-4dff-be54-90aed01472df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:10.395720 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a471b79-0456-4881-bfb8-65a67bdfacd1 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:10.417925 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:10.431412 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348bda77-0188-4327-adbc-4e2228ef773d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:10.432519 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bda77-0188-4327-adbc-4e2228ef773d None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:10.432519 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bda77-0188-4327-adbc-4e2228ef773d None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:10.432730 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bda77-0188-4327-adbc-4e2228ef773d None admin] PATH_INFO: `/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:10.433777 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348bda77-0188-4327-adbc-4e2228ef773d None admin] RBAC: Authorizing `identity:get_project(project_id=c60b6aa5591d4178ba9eb5fd05b7ee73)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:10.435243 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348bda77-0188-4327-adbc-4e2228ef773d None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:10.436897 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 42/83] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:10 2022] GET /identity/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:10.437668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348bda77-0188-4327-adbc-4e2228ef773d None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:10.467858 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a781f3c1-49b8-4dff-be54-90aed01472df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:10.469359 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a781f3c1-49b8-4dff-be54-90aed01472df None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:10.502839 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:10.536261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348bda77-0188-4327-adbc-4e2228ef773d None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:10.565838 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 42/84] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:34:10.566781 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a781f3c1-49b8-4dff-be54-90aed01472df None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:10.570800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627741c4-6895-4cb9-818b-0430bbe54c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:10.571438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627741c4-6895-4cb9-818b-0430bbe54c68 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:10.571723 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627741c4-6895-4cb9-818b-0430bbe54c68 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:10.572000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627741c4-6895-4cb9-818b-0430bbe54c68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:10.573141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627741c4-6895-4cb9-818b-0430bbe54c68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:10.574587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627741c4-6895-4cb9-818b-0430bbe54c68 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:10.576141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 43/85] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:34:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:10.576923 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627741c4-6895-4cb9-818b-0430bbe54c68 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:10.617295 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:10.665247 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a781f3c1-49b8-4dff-be54-90aed01472df None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:10.674787 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627741c4-6895-4cb9-818b-0430bbe54c68 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:10.697534 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a517b99-c14e-4a8d-9b04-af7eee8767d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:10.698150 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a517b99-c14e-4a8d-9b04-af7eee8767d9 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:10.698436 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a517b99-c14e-4a8d-9b04-af7eee8767d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:10.698717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a517b99-c14e-4a8d-9b04-af7eee8767d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:10.707065 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af176f4a-c0ab-48fa-8dcd-dc539f1e64a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:10.707669 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af176f4a-c0ab-48fa-8dcd-dc539f1e64a2 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:10.707953 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af176f4a-c0ab-48fa-8dcd-dc539f1e64a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:10.708230 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af176f4a-c0ab-48fa-8dcd-dc539f1e64a2 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:10.708932 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af176f4a-c0ab-48fa-8dcd-dc539f1e64a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:10.709238 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af176f4a-c0ab-48fa-8dcd-dc539f1e64a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:10.710617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af176f4a-c0ab-48fa-8dcd-dc539f1e64a2 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:10.711935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:10.719326 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 44/86] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:34:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:10.724617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af176f4a-c0ab-48fa-8dcd-dc539f1e64a2 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:10.766420 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a517b99-c14e-4a8d-9b04-af7eee8767d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:10.768399 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a517b99-c14e-4a8d-9b04-af7eee8767d9 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:10.816071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:10.821058 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af176f4a-c0ab-48fa-8dcd-dc539f1e64a2 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:10.854987 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9513acfc-17e2-4030-bc02-c14e755c5641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:10.855601 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513acfc-17e2-4030-bc02-c14e755c5641 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:10.855886 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513acfc-17e2-4030-bc02-c14e755c5641 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:10.856163 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513acfc-17e2-4030-bc02-c14e755c5641 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:10.856926 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9513acfc-17e2-4030-bc02-c14e755c5641 None admin] RBAC: Adding query filter params (role.id=5032636d8729461c9926a59b9bcd041b, scope.project.id=c60b6aa5591d4178ba9eb5fd05b7ee73, user.id=b92a689fcc4842b3aa1d3a5c3f5e47b4) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:10.857233 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9513acfc-17e2-4030-bc02-c14e755c5641 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:10.858696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9513acfc-17e2-4030-bc02-c14e755c5641 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:10.866316 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 43/87] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:34:10.871274 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 45/88] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:34:10 2022] GET /identity/v3/role_assignments?user.id=b92a689fcc4842b3aa1d3a5c3f5e47b4&scope.project.id=c60b6aa5591d4178ba9eb5fd05b7ee73&role.id=5032636d8729461c9926a59b9bcd041b => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:10.871378 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a517b99-c14e-4a8d-9b04-af7eee8767d9 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:10.871462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9513acfc-17e2-4030-bc02-c14e755c5641 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:10.948784 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:10.996731 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9513acfc-17e2-4030-bc02-c14e755c5641 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:11.001871 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a517b99-c14e-4a8d-9b04-af7eee8767d9 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:11.042071 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe3b11-8729-451a-ac80-aef7c435ac1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:11.042559 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe3b11-8729-451a-ac80-aef7c435ac1d None admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:11.042843 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe3b11-8729-451a-ac80-aef7c435ac1d None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:11.043122 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe3b11-8729-451a-ac80-aef7c435ac1d None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:11.043822 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe3b11-8729-451a-ac80-aef7c435ac1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:11.045322 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe3b11-8729-451a-ac80-aef7c435ac1d None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:11.047038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004cf331-8e43-485c-8b19-d2975e8715be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:11.047677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004cf331-8e43-485c-8b19-d2975e8715be None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:11.047958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004cf331-8e43-485c-8b19-d2975e8715be None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:11.048230 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004cf331-8e43-485c-8b19-d2975e8715be None admin] PATH_INFO: `/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:11.049401 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004cf331-8e43-485c-8b19-d2975e8715be None admin] RBAC: Authorizing `identity:get_project(project_id=c60b6aa5591d4178ba9eb5fd05b7ee73)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:11.050825 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004cf331-8e43-485c-8b19-d2975e8715be None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:11.057541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 44/89] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:10 2022] GET /identity/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:11.057541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004cf331-8e43-485c-8b19-d2975e8715be None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:11.076632 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:11.153699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004cf331-8e43-485c-8b19-d2975e8715be None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:11.159360 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 46/90] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Aug 26 00:34:10 2022] POST /identity/v3/users => generated 289 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:34:11.160379 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe3b11-8729-451a-ac80-aef7c435ac1d None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:11.193910 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d41261-f575-4206-a580-f69df9c2a2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:11.194572 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d41261-f575-4206-a580-f69df9c2a2e6 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:11.194858 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d41261-f575-4206-a580-f69df9c2a2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:11.195139 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d41261-f575-4206-a580-f69df9c2a2e6 None admin] PATH_INFO: `/v3/users/b92a689fcc4842b3aa1d3a5c3f5e47b4` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:11.196387 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d41261-f575-4206-a580-f69df9c2a2e6 None admin] RBAC: Authorizing `identity:get_user(user_id=b92a689fcc4842b3aa1d3a5c3f5e47b4)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:11.198407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d41261-f575-4206-a580-f69df9c2a2e6 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:11.199942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 45/91] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:11 2022] GET /identity/v3/users/b92a689fcc4842b3aa1d3a5c3f5e47b4 => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:11.200855 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d41261-f575-4206-a580-f69df9c2a2e6 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:11.307256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:11.313646 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe3b11-8729-451a-ac80-aef7c435ac1d None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:11.327672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d41261-f575-4206-a580-f69df9c2a2e6 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:11.337820 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:11.366812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691eae3-24d6-4c3e-b819-8f4a7be86d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:11.367479 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691eae3-24d6-4c3e-b819-8f4a7be86d0f None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:11.367767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691eae3-24d6-4c3e-b819-8f4a7be86d0f None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:11.368043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691eae3-24d6-4c3e-b819-8f4a7be86d0f None admin] PATH_INFO: `/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:11.369689 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691eae3-24d6-4c3e-b819-8f4a7be86d0f None admin] RBAC: Authorizing `identity:get_project(project_id=c60b6aa5591d4178ba9eb5fd05b7ee73)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:11.370283 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ca575-e45d-4855-8ffa-dae11cb0cc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:11.371193 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ca575-e45d-4855-8ffa-dae11cb0cc5f None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:11.371659 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ca575-e45d-4855-8ffa-dae11cb0cc5f None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:11.371860 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691eae3-24d6-4c3e-b819-8f4a7be86d0f None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:11.373092 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ca575-e45d-4855-8ffa-dae11cb0cc5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:11.374020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918ca575-e45d-4855-8ffa-dae11cb0cc5f None admin] RBAC: Adding query filter params (role.id=1a33e3c8cb8347feb9498595886cc621, scope.project.id=c60b6aa5591d4178ba9eb5fd05b7ee73, user.id=795669ccac0f433b900784c76b5b0142) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:11.374517 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918ca575-e45d-4855-8ffa-dae11cb0cc5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:11.374680 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 47/92] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:11 2022] GET /identity/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:11.380065 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691eae3-24d6-4c3e-b819-8f4a7be86d0f None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:11.380213 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918ca575-e45d-4855-8ffa-dae11cb0cc5f None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:11.386148 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 46/93] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:34:11 2022] GET /identity/v3/role_assignments?user.id=795669ccac0f433b900784c76b5b0142&scope.project.id=c60b6aa5591d4178ba9eb5fd05b7ee73&role.id=1a33e3c8cb8347feb9498595886cc621 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:11.387363 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ca575-e45d-4855-8ffa-dae11cb0cc5f None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:11.511982 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691eae3-24d6-4c3e-b819-8f4a7be86d0f None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:11.552671 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506716df-c407-4dff-9f6d-62c25041ecf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:11.553304 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506716df-c407-4dff-9f6d-62c25041ecf9 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:11.553588 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506716df-c407-4dff-9f6d-62c25041ecf9 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:11.553866 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506716df-c407-4dff-9f6d-62c25041ecf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:11.554380 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ca575-e45d-4855-8ffa-dae11cb0cc5f None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:11.606773 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96d8b9-3ef8-43c1-8147-0612c6ef507c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:11.607379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96d8b9-3ef8-43c1-8147-0612c6ef507c None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:11.607660 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96d8b9-3ef8-43c1-8147-0612c6ef507c None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:11.607931 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96d8b9-3ef8-43c1-8147-0612c6ef507c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:11.608866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e96d8b9-3ef8-43c1-8147-0612c6ef507c None admin] RBAC: Adding query filter params (role.id=423aa6ac8b05402da7587e83e5c4ce09, scope.project.id=c60b6aa5591d4178ba9eb5fd05b7ee73, user.id=b92a689fcc4842b3aa1d3a5c3f5e47b4) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:11.609128 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e96d8b9-3ef8-43c1-8147-0612c6ef507c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:11.610586 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e96d8b9-3ef8-43c1-8147-0612c6ef507c None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:11.617105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 47/94] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:34:11 2022] GET /identity/v3/role_assignments?user.id=b92a689fcc4842b3aa1d3a5c3f5e47b4&scope.project.id=c60b6aa5591d4178ba9eb5fd05b7ee73&role.id=423aa6ac8b05402da7587e83e5c4ce09 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:11.631924 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:11.654461 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506716df-c407-4dff-9f6d-62c25041ecf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:11.656024 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506716df-c407-4dff-9f6d-62c25041ecf9 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:11.779743 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 48/95] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:34:11.785247 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96d8b9-3ef8-43c1-8147-0612c6ef507c None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:11.807990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:11.906981 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96d8b9-3ef8-43c1-8147-0612c6ef507c None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:11.945028 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80941e-ff18-486e-8d44-db1e65ab32ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:11.945683 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80941e-ff18-486e-8d44-db1e65ab32ba None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:11.945970 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80941e-ff18-486e-8d44-db1e65ab32ba None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:11.946252 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80941e-ff18-486e-8d44-db1e65ab32ba None admin] PATH_INFO: `/v3/projects/d7c37d24ebc147969c92537604aabc6b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:11.947442 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d80941e-ff18-486e-8d44-db1e65ab32ba None admin] RBAC: Authorizing `identity:get_project(project_id=d7c37d24ebc147969c92537604aabc6b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:11.948957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d80941e-ff18-486e-8d44-db1e65ab32ba None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:11.951174 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 48/96] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:11 2022] GET /identity/v3/projects/d7c37d24ebc147969c92537604aabc6b => generated 312 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:11.957088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506716df-c407-4dff-9f6d-62c25041ecf9 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:12.063497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:12.087953 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506716df-c407-4dff-9f6d-62c25041ecf9 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:12.129118 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c446519-e9ef-4c38-b343-ccad75d686ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:12.129741 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c446519-e9ef-4c38-b343-ccad75d686ba None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:12.130027 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c446519-e9ef-4c38-b343-ccad75d686ba None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:12.130313 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c446519-e9ef-4c38-b343-ccad75d686ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:12.131048 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c446519-e9ef-4c38-b343-ccad75d686ba None admin] RBAC: Adding query filter params (role.id=5032636d8729461c9926a59b9bcd041b, scope.project.id=d7c37d24ebc147969c92537604aabc6b, user.id=b92a689fcc4842b3aa1d3a5c3f5e47b4) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:12.131361 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c446519-e9ef-4c38-b343-ccad75d686ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:12.132908 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c446519-e9ef-4c38-b343-ccad75d686ba None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:12.145702 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 49/97] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:34:11 2022] GET /identity/v3/role_assignments?user.id=b92a689fcc4842b3aa1d3a5c3f5e47b4&scope.project.id=d7c37d24ebc147969c92537604aabc6b&role.id=5032636d8729461c9926a59b9bcd041b => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:13.257748 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb18094-84ca-436b-8b08-95f50edd5c59 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:13.258767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb18094-84ca-436b-8b08-95f50edd5c59 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:13.259304 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb18094-84ca-436b-8b08-95f50edd5c59 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:13.261116 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 49/98] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:13.266747 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc0f2f-802b-4efe-bda5-77ed117c0ff1 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:13.267549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc0f2f-802b-4efe-bda5-77ed117c0ff1 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:13.268603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc0f2f-802b-4efe-bda5-77ed117c0ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:13.291462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-bcdc0f2f-802b-4efe-bda5-77ed117c0ff1 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:13.293230 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-bcdc0f2f-802b-4efe-bda5-77ed117c0ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:13.374794 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 50/99] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:13.929974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb18094-84ca-436b-8b08-95f50edd5c59 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:14.029697 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:14.053184 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb18094-84ca-436b-8b08-95f50edd5c59 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:14.100511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b1384-af03-4072-8e48-9c9a30a47366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:14.101560 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b1384-af03-4072-8e48-9c9a30a47366 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:14.102044 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b1384-af03-4072-8e48-9c9a30a47366 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:14.102590 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b1384-af03-4072-8e48-9c9a30a47366 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:14.121379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b1384-af03-4072-8e48-9c9a30a47366 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:14.126556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-943b1384-af03-4072-8e48-9c9a30a47366 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:14.126556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:14.126556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:14.126556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:14.126556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:14.126556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:14.126556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:14.126556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:14.126556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:14.126556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:14.126556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:14.126556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:14.126556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:14.126556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:14.126556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:14.126556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:14.127456 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:14.127456 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:14.127456 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:14.127456 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:14.127456 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:14.127456 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:14.127456 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:14.127456 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:14.127456 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:14.127456 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:14.127456 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:14.127456 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:14.127456 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:14.127456 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:14.127456 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:14.127456 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:14.128235 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:14.128235 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:14.128235 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:14.128235 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:14.128235 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:14.128235 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:14.128235 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:14.128235 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:14.128235 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:14.128235 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:14.128235 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:14.128235 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:34:14.130210 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 50/100] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:14.136608 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc0f2f-802b-4efe-bda5-77ed117c0ff1 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:14.191299 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592141f-9829-4117-a83e-903833e0a332 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:14.191912 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592141f-9829-4117-a83e-903833e0a332 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:14.192394 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592141f-9829-4117-a83e-903833e0a332 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:14.194532 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 51/101] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:34:14.200052 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e92a5-dd10-4182-a864-a85a3050a879 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:14.200661 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e92a5-dd10-4182-a864-a85a3050a879 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:14.201211 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e92a5-dd10-4182-a864-a85a3050a879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:14.225788 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-ae6e92a5-dd10-4182-a864-a85a3050a879 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:14.226885 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-ae6e92a5-dd10-4182-a864-a85a3050a879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:14.248900 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:14.271578 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc0f2f-802b-4efe-bda5-77ed117c0ff1 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:14.311341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 52/102] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:34:14.312213 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aade134-7612-401d-9f15-c2bdacb39256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:14.313027 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aade134-7612-401d-9f15-c2bdacb39256 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:14.313304 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1296b1-d5bb-4b16-8bdd-d793b5f7e292 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:14.313667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aade134-7612-401d-9f15-c2bdacb39256 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:14.313778 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1296b1-d5bb-4b16-8bdd-d793b5f7e292 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:14.314125 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1296b1-d5bb-4b16-8bdd-d793b5f7e292 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:14.314320 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aade134-7612-401d-9f15-c2bdacb39256 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:14.315220 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aade134-7612-401d-9f15-c2bdacb39256 None admin] RBAC: Adding query filter params (name=member) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:14.315861 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 53/103] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:14.316011 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aade134-7612-401d-9f15-c2bdacb39256 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:14.317613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aade134-7612-401d-9f15-c2bdacb39256 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:14.321401 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907da5d-b319-4407-947c-ddd66a8c5fbc None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:14.321797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907da5d-b319-4407-947c-ddd66a8c5fbc None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:14.322175 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907da5d-b319-4407-947c-ddd66a8c5fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:14.327808 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 51/104] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:34:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:14.346528 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aade134-7612-401d-9f15-c2bdacb39256 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:14.348206 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-c907da5d-b319-4407-947c-ddd66a8c5fbc None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:14.349617 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-c907da5d-b319-4407-947c-ddd66a8c5fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:14.434367 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 54/105] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:34:14.434570 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33438d-ba04-4bf8-b1e7-64dc31c47c55 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:14.434862 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33438d-ba04-4bf8-b1e7-64dc31c47c55 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:14.435150 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33438d-ba04-4bf8-b1e7-64dc31c47c55 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:14.436443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 55/106] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:14.444678 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2121753-6647-492e-b824-9e49a473d6ad None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:14.447468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2121753-6647-492e-b824-9e49a473d6ad None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:14.447468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2121753-6647-492e-b824-9e49a473d6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:14.456965 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:14.478268 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-a2121753-6647-492e-b824-9e49a473d6ad None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:14.479472 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-a2121753-6647-492e-b824-9e49a473d6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:14.499986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aade134-7612-401d-9f15-c2bdacb39256 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:14.570893 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f2b86-023a-4353-80a5-900dbafca1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:14.571559 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f2b86-023a-4353-80a5-900dbafca1d0 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:14.571843 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f2b86-023a-4353-80a5-900dbafca1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:14.572126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f2b86-023a-4353-80a5-900dbafca1d0 None admin] PATH_INFO: `/v3/users/b92a689fcc4842b3aa1d3a5c3f5e47b4` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:14.573372 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f2b86-023a-4353-80a5-900dbafca1d0 None admin] RBAC: Authorizing `identity:get_user(user_id=b92a689fcc4842b3aa1d3a5c3f5e47b4)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:14.575646 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f2b86-023a-4353-80a5-900dbafca1d0 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:14.577274 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 52/107] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:14 2022] GET /identity/v3/users/b92a689fcc4842b3aa1d3a5c3f5e47b4 => generated 275 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:14.594831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f2b86-023a-4353-80a5-900dbafca1d0 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:14.613821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:14.626174 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 56/108] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:14.700064 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2121753-6647-492e-b824-9e49a473d6ad None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:14.729675 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f2b86-023a-4353-80a5-900dbafca1d0 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:14.785781 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1da0ee-fc61-49b5-b4da-12554847644b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:14.792497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1da0ee-fc61-49b5-b4da-12554847644b None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:14.793362 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1da0ee-fc61-49b5-b4da-12554847644b None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:14.794080 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1da0ee-fc61-49b5-b4da-12554847644b None admin] PATH_INFO: `/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:14.795623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1da0ee-fc61-49b5-b4da-12554847644b None admin] RBAC: Authorizing `identity:get_project(project_id=c60b6aa5591d4178ba9eb5fd05b7ee73)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:14.797618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1da0ee-fc61-49b5-b4da-12554847644b None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:14.799686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 53/109] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:14 2022] GET /identity/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:14.816562 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61e9cb-dfe4-4fd1-95b2-bda0b72a25f9 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:14.817351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61e9cb-dfe4-4fd1-95b2-bda0b72a25f9 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:14.818139 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61e9cb-dfe4-4fd1-95b2-bda0b72a25f9 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:14.819677 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 54/110] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:34:14.820709 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61e9cb-dfe4-4fd1-95b2-bda0b72a25f9 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:14.824558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2121753-6647-492e-b824-9e49a473d6ad None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:14.857744 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:14.863155 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c4650-1fbe-47a9-9e74-7b737aa4f844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:14.863790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c4650-1fbe-47a9-9e74-7b737aa4f844 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:14.864082 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c4650-1fbe-47a9-9e74-7b737aa4f844 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:14.864364 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c4650-1fbe-47a9-9e74-7b737aa4f844 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:14.876538 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c4650-1fbe-47a9-9e74-7b737aa4f844 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:14.882362 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-855c4650-1fbe-47a9-9e74-7b737aa4f844 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:34:14.882362 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:14.882362 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:14.882362 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:14.882362 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:14.882362 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:14.882362 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:14.882362 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:14.882362 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:14.882362 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:14.882362 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:14.882362 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:14.882362 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:14.882362 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:14.882362 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:14.882362 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:14.883372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:14.883372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:14.883372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:14.883372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:14.883372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:14.883372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:14.883372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:14.883372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:14.883372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:14.883372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:14.883372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:14.883372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:14.883372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:14.883372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:14.883372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:14.883372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:14.884160 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:14.884160 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:14.884160 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:14.884160 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:14.884160 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:14.884160 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:14.884160 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:14.884160 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:14.884160 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:14.884160 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:14.884160 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:34:14.884160 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:34:14.884160 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 57/111] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:34:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:34:14.888495 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb58aaf-25d1-4b95-b7ea-9d278d3499db None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:14.888495 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb58aaf-25d1-4b95-b7ea-9d278d3499db None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:14.888495 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb58aaf-25d1-4b95-b7ea-9d278d3499db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:14.907454 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-5cb58aaf-25d1-4b95-b7ea-9d278d3499db None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:14.908935 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-5cb58aaf-25d1-4b95-b7ea-9d278d3499db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:14.935812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:14.961896 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61e9cb-dfe4-4fd1-95b2-bda0b72a25f9 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:14.985059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 58/112] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:14.986329 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb58aaf-25d1-4b95-b7ea-9d278d3499db None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:15.006555 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec8c1f-989b-4329-b8da-a7907e0a64f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:15.007244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec8c1f-989b-4329-b8da-a7907e0a64f0 None admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:15.007525 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec8c1f-989b-4329-b8da-a7907e0a64f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:15.007803 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec8c1f-989b-4329-b8da-a7907e0a64f0 None admin] PATH_INFO: `/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73/users/b92a689fcc4842b3aa1d3a5c3f5e47b4/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:15.010324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec8c1f-989b-4329-b8da-a7907e0a64f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=c60b6aa5591d4178ba9eb5fd05b7ee73, user_id=b92a689fcc4842b3aa1d3a5c3f5e47b4, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:15.012113 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec8c1f-989b-4329-b8da-a7907e0a64f0 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:15.076576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 55/113] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Aug 26 00:34:14 2022] PUT /identity/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73/users/b92a689fcc4842b3aa1d3a5c3f5e47b4/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:34:15.083677 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec8c1f-989b-4329-b8da-a7907e0a64f0 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:15.169711 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb58aaf-25d1-4b95-b7ea-9d278d3499db None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:15.224950 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:15.252934 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913109c6-006e-40a4-96c8-51454c951b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:15.253589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913109c6-006e-40a4-96c8-51454c951b7b None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:15.253872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913109c6-006e-40a4-96c8-51454c951b7b None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:15.254143 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913109c6-006e-40a4-96c8-51454c951b7b None admin] PATH_INFO: `/v3/domains/default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:15.255305 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913109c6-006e-40a4-96c8-51454c951b7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:15.257354 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913109c6-006e-40a4-96c8-51454c951b7b None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:15.259439 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 59/114] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:34:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:15.260740 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913109c6-006e-40a4-96c8-51454c951b7b None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:15.293028 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec8c1f-989b-4329-b8da-a7907e0a64f0 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:15.334670 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3c2e0-49a4-4b55-a7f8-b9796c4381c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:15.335296 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3c2e0-49a4-4b55-a7f8-b9796c4381c5 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:15.335581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3c2e0-49a4-4b55-a7f8-b9796c4381c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:15.335863 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3c2e0-49a4-4b55-a7f8-b9796c4381c5 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:15.336569 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af3c2e0-49a4-4b55-a7f8-b9796c4381c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:15.336879 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af3c2e0-49a4-4b55-a7f8-b9796c4381c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:15.338299 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af3c2e0-49a4-4b55-a7f8-b9796c4381c5 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:15.340790 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:15.349566 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 56/115] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:15.350421 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3c2e0-49a4-4b55-a7f8-b9796c4381c5 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:15.394461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913109c6-006e-40a4-96c8-51454c951b7b None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:15.450004 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30386204-b0d2-4c89-8872-9b9175dda9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:15.450652 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30386204-b0d2-4c89-8872-9b9175dda9a1 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:15.450935 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30386204-b0d2-4c89-8872-9b9175dda9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:15.451209 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30386204-b0d2-4c89-8872-9b9175dda9a1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:15.467051 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30386204-b0d2-4c89-8872-9b9175dda9a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:15.471190 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3c2e0-49a4-4b55-a7f8-b9796c4381c5 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:15.475807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-30386204-b0d2-4c89-8872-9b9175dda9a1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 00:34:15.475807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:15.475807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:15.475807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:15.475807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:15.475807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:15.475807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:15.475807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:15.475807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:15.475807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:15.475807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:15.475807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:15.475807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:15.475807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:15.475807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:15.475807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:15.476682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:15.476682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:15.476682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:15.476682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:15.476682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:15.476682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:15.476682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:15.476682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:15.476682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:15.476682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:15.476682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:15.476682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:15.476682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:15.476682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:15.476682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:15.476682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:15.477476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:15.477476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:15.477476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:15.477476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:15.477476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:15.477476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:15.477476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:15.477476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:15.477476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:15.477476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:15.477476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 00:34:15.477476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:34:15.480430 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 60/116] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:15.481616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30386204-b0d2-4c89-8872-9b9175dda9a1 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:15.515940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd720d-9d64-4b8e-a044-141083c57f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:15.516607 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd720d-9d64-4b8e-a044-141083c57f20 None admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:15.516942 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd720d-9d64-4b8e-a044-141083c57f20 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:15.517220 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd720d-9d64-4b8e-a044-141083c57f20 None admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:15.518451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd720d-9d64-4b8e-a044-141083c57f20 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:15.519908 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd720d-9d64-4b8e-a044-141083c57f20 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:15.550746 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:15.601145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30386204-b0d2-4c89-8872-9b9175dda9a1 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:15.618069 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 57/117] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Aug 26 00:34:15 2022] POST /identity/v3/projects => generated 302 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:34:15.618914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd720d-9d64-4b8e-a044-141083c57f20 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:15.652004 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e858d-50fc-469f-a15f-e83b522e5e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:15.653290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e858d-50fc-469f-a15f-e83b522e5e19 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:15.653836 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e858d-50fc-469f-a15f-e83b522e5e19 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:15.654109 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e858d-50fc-469f-a15f-e83b522e5e19 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:15.667814 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626e858d-50fc-469f-a15f-e83b522e5e19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:15.673789 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-626e858d-50fc-469f-a15f-e83b522e5e19 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 00:34:15.673789 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:15.673789 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:15.673789 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:15.673789 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:15.673789 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:15.673789 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:15.673789 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:15.673789 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:15.673789 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:15.673789 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:15.673789 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:15.673789 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:15.673789 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:15.673789 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:15.673789 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:15.675020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:15.675020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:15.675020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:15.675020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:15.675020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:15.675020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:15.675020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:15.675020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:15.675020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:15.675020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:15.675020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:15.675020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:15.675020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:15.675020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:15.675020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:15.675020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:15.675818 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:15.675818 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:15.675818 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:15.675818 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:15.675818 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:15.675818 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:15.675818 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:15.675818 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:15.675818 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:15.675818 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:15.675818 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 00:34:15.675818 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:34:15.675818 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 61/118] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Aug 26 00:34:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:15.681059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e858d-50fc-469f-a15f-e83b522e5e19 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:15.692443 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:15.765606 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd720d-9d64-4b8e-a044-141083c57f20 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:15.802746 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e858d-50fc-469f-a15f-e83b522e5e19 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:15.811514 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b293a0b0-9969-4c3f-ad8c-b9d6d9653ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:15.812722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b293a0b0-9969-4c3f-ad8c-b9d6d9653ff4 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:15.813016 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b293a0b0-9969-4c3f-ad8c-b9d6d9653ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:15.813300 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b293a0b0-9969-4c3f-ad8c-b9d6d9653ff4 None admin] PATH_INFO: `/v3/users/795669ccac0f433b900784c76b5b0142` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:15.815794 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b293a0b0-9969-4c3f-ad8c-b9d6d9653ff4 None admin] RBAC: Authorizing `identity:get_user(user_id=795669ccac0f433b900784c76b5b0142)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:15.817948 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:15.818151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b293a0b0-9969-4c3f-ad8c-b9d6d9653ff4 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:15.820900 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 58/119] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:15 2022] GET /identity/v3/users/795669ccac0f433b900784c76b5b0142 => generated 247 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:15.828400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb95e41-7501-4c19-a69c-00fc5ac68c13 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:15.828400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb95e41-7501-4c19-a69c-00fc5ac68c13 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:15.828400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb95e41-7501-4c19-a69c-00fc5ac68c13 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:15.828400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 59/120] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:15.828400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb95e41-7501-4c19-a69c-00fc5ac68c13 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:15.853338 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848bca0b-2dfc-4a44-aa9c-bb714edeeeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:15.854319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bca0b-2dfc-4a44-aa9c-bb714edeeeb1 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:15.855356 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bca0b-2dfc-4a44-aa9c-bb714edeeeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:15.855843 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bca0b-2dfc-4a44-aa9c-bb714edeeeb1 None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:15.857125 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848bca0b-2dfc-4a44-aa9c-bb714edeeeb1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:15.857877 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848bca0b-2dfc-4a44-aa9c-bb714edeeeb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:15.860214 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848bca0b-2dfc-4a44-aa9c-bb714edeeeb1 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:15.874449 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 62/121] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:34:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:15.876539 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7dcf4-d73c-4546-b06d-39ec3a5e25fa None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:15.877043 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7dcf4-d73c-4546-b06d-39ec3a5e25fa None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:15.877498 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7dcf4-d73c-4546-b06d-39ec3a5e25fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:15.902781 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-91f7dcf4-d73c-4546-b06d-39ec3a5e25fa None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:15.904558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-91f7dcf4-d73c-4546-b06d-39ec3a5e25fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:15.945728 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb95e41-7501-4c19-a69c-00fc5ac68c13 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:15.965655 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:15.984186 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca11f2c6-3cea-4eaa-a427-b86c6fefd304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:15.984960 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca11f2c6-3cea-4eaa-a427-b86c6fefd304 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:15.985350 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca11f2c6-3cea-4eaa-a427-b86c6fefd304 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:15.985736 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca11f2c6-3cea-4eaa-a427-b86c6fefd304 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:15.986643 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca11f2c6-3cea-4eaa-a427-b86c6fefd304 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:15.987022 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca11f2c6-3cea-4eaa-a427-b86c6fefd304 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:15.988570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca11f2c6-3cea-4eaa-a427-b86c6fefd304 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:15.993519 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 63/122] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:15.999443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f7dcf4-d73c-4546-b06d-39ec3a5e25fa None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:15.999591 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 60/123] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:34:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:15.999591 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca11f2c6-3cea-4eaa-a427-b86c6fefd304 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:16.122175 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f7dcf4-d73c-4546-b06d-39ec3a5e25fa None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:16.124020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:16.162278 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4db16-45bd-4e6e-9dd2-55cb32f7e61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:16.163322 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4db16-45bd-4e6e-9dd2-55cb32f7e61a None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:16.163814 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4db16-45bd-4e6e-9dd2-55cb32f7e61a None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:16.164288 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4db16-45bd-4e6e-9dd2-55cb32f7e61a None admin] PATH_INFO: `/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:16.165644 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd4db16-45bd-4e6e-9dd2-55cb32f7e61a None admin] RBAC: Authorizing `identity:get_project(project_id=c60b6aa5591d4178ba9eb5fd05b7ee73)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:16.167295 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd4db16-45bd-4e6e-9dd2-55cb32f7e61a None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:16.169502 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 64/124] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:15 2022] GET /identity/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:16.170504 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4db16-45bd-4e6e-9dd2-55cb32f7e61a None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:16.197099 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca11f2c6-3cea-4eaa-a427-b86c6fefd304 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:16.268399 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e110da-7c40-4eb4-a2d9-d379393f4a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:16.270074 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e110da-7c40-4eb4-a2d9-d379393f4a0d None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:16.270736 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e110da-7c40-4eb4-a2d9-d379393f4a0d None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:16.272335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e110da-7c40-4eb4-a2d9-d379393f4a0d None admin] PATH_INFO: `/v3/users/f52fa6565d874f4e9d40b577d033f854` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:16.303473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e110da-7c40-4eb4-a2d9-d379393f4a0d None admin] RBAC: Authorizing `identity:get_user(user_id=f52fa6565d874f4e9d40b577d033f854)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:16.307089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e110da-7c40-4eb4-a2d9-d379393f4a0d None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:16.309337 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 61/125] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:15 2022] GET /identity/v3/users/f52fa6565d874f4e9d40b577d033f854 => generated 289 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:16.311001 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e110da-7c40-4eb4-a2d9-d379393f4a0d None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:16.349033 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4db16-45bd-4e6e-9dd2-55cb32f7e61a None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:16.386698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:16.412033 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fcf2c-df86-456e-b759-8f0112b97eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:16.412719 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fcf2c-df86-456e-b759-8f0112b97eed None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:16.413007 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fcf2c-df86-456e-b759-8f0112b97eed None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:16.413283 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fcf2c-df86-456e-b759-8f0112b97eed None admin] PATH_INFO: `/v3/users/b92a689fcc4842b3aa1d3a5c3f5e47b4` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:16.414439 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03fcf2c-df86-456e-b759-8f0112b97eed None admin] RBAC: Authorizing `identity:get_user(user_id=b92a689fcc4842b3aa1d3a5c3f5e47b4)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:16.416754 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03fcf2c-df86-456e-b759-8f0112b97eed None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:16.419202 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 65/126] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:16 2022] GET /identity/v3/users/b92a689fcc4842b3aa1d3a5c3f5e47b4 => generated 275 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:16.420322 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fcf2c-df86-456e-b759-8f0112b97eed None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:16.473433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:16.480343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e110da-7c40-4eb4-a2d9-d379393f4a0d None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:16.528903 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb04e7-1514-4c6b-8b8b-8abd5d5fd580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:16.529988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb04e7-1514-4c6b-8b8b-8abd5d5fd580 None admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:16.530512 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb04e7-1514-4c6b-8b8b-8abd5d5fd580 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:16.530988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb04e7-1514-4c6b-8b8b-8abd5d5fd580 None admin] PATH_INFO: `/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73/users/795669ccac0f433b900784c76b5b0142/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:16.533707 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb04e7-1514-4c6b-8b8b-8abd5d5fd580 None admin] RBAC: Authorizing `identity:create_grant(project_id=c60b6aa5591d4178ba9eb5fd05b7ee73, user_id=795669ccac0f433b900784c76b5b0142, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:16.536025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb04e7-1514-4c6b-8b8b-8abd5d5fd580 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:16.545902 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fcf2c-df86-456e-b759-8f0112b97eed None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:16.583308 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300345b-417b-4fe2-847b-4db98ef056b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:16.584125 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300345b-417b-4fe2-847b-4db98ef056b4 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:16.584413 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300345b-417b-4fe2-847b-4db98ef056b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:16.584730 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300345b-417b-4fe2-847b-4db98ef056b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:16.637273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 62/127] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Aug 26 00:34:16 2022] PUT /identity/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73/users/795669ccac0f433b900784c76b5b0142/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:34:16.642388 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb04e7-1514-4c6b-8b8b-8abd5d5fd580 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:16.697674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300345b-417b-4fe2-847b-4db98ef056b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:16.699319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300345b-417b-4fe2-847b-4db98ef056b4 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:16.782987 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:16.789048 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb04e7-1514-4c6b-8b8b-8abd5d5fd580 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:16.813395 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:16.837541 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999659ab-b226-466c-afa9-7f534ec77eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:16.838602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999659ab-b226-466c-afa9-7f534ec77eb5 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:16.839133 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999659ab-b226-466c-afa9-7f534ec77eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:16.839613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999659ab-b226-466c-afa9-7f534ec77eb5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:16.853301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999659ab-b226-466c-afa9-7f534ec77eb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:16.860922 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-999659ab-b226-466c-afa9-7f534ec77eb5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:16.860922 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:16.860922 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:16.860922 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:16.860922 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:16.860922 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:16.860922 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:16.860922 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:16.860922 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:16.860922 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:16.860922 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:16.860922 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:16.860922 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:16.860922 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:16.860922 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:16.860922 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:16.862460 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:16.862460 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:16.862460 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:16.862460 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:16.862460 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:16.862460 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:16.862460 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:16.862460 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:16.862460 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:16.862460 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:16.862460 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:16.862460 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:16.862460 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:16.862460 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:16.862460 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:16.862460 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:16.863273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:16.863273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:16.863273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:16.863273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:16.863273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:16.863273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:16.863273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:16.863273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:16.863273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:16.863273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:16.863273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:16.863273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:34:16.865983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 63/128] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:16.867497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999659ab-b226-466c-afa9-7f534ec77eb5 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:16.919667 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 66/129] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:34:16.920672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300345b-417b-4fe2-847b-4db98ef056b4 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:16.990084 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999659ab-b226-466c-afa9-7f534ec77eb5 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:17.026033 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58119639-9e7c-474b-8b76-135e2b17b10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:17.026672 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58119639-9e7c-474b-8b76-135e2b17b10f None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:17.026962 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58119639-9e7c-474b-8b76-135e2b17b10f None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:17.027244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58119639-9e7c-474b-8b76-135e2b17b10f None admin] PATH_INFO: `/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:17.028730 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58119639-9e7c-474b-8b76-135e2b17b10f None admin] RBAC: Authorizing `identity:get_project(project_id=c60b6aa5591d4178ba9eb5fd05b7ee73)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:17.030170 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58119639-9e7c-474b-8b76-135e2b17b10f None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:17.032535 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 64/130] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:16 2022] GET /identity/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:17.033966 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58119639-9e7c-474b-8b76-135e2b17b10f None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:17.043700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300345b-417b-4fe2-847b-4db98ef056b4 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:17.065109 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:17.083225 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861dc0ac-8f60-4cc8-bca4-e9369fe0f475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:17.084230 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861dc0ac-8f60-4cc8-bca4-e9369fe0f475 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:17.084762 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861dc0ac-8f60-4cc8-bca4-e9369fe0f475 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:17.085239 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861dc0ac-8f60-4cc8-bca4-e9369fe0f475 None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:17.086131 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861dc0ac-8f60-4cc8-bca4-e9369fe0f475 None admin] RBAC: Adding query filter params (name=member) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:17.086687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861dc0ac-8f60-4cc8-bca4-e9369fe0f475 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:17.088299 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861dc0ac-8f60-4cc8-bca4-e9369fe0f475 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:17.094211 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:17.099748 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 67/131] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:34:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:17.102013 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861dc0ac-8f60-4cc8-bca4-e9369fe0f475 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:17.147602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58119639-9e7c-474b-8b76-135e2b17b10f None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:17.184097 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96db62-4a58-4888-801f-4b2ecfb85035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:17.184097 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96db62-4a58-4888-801f-4b2ecfb85035 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:17.184352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96db62-4a58-4888-801f-4b2ecfb85035 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:17.184433 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96db62-4a58-4888-801f-4b2ecfb85035 None admin] PATH_INFO: `/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:17.185624 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff96db62-4a58-4888-801f-4b2ecfb85035 None admin] RBAC: Authorizing `identity:get_project(project_id=c60b6aa5591d4178ba9eb5fd05b7ee73)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:17.187083 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff96db62-4a58-4888-801f-4b2ecfb85035 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:17.189099 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 65/132] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:17 2022] GET /identity/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:17.190991 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96db62-4a58-4888-801f-4b2ecfb85035 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:17.219046 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861dc0ac-8f60-4cc8-bca4-e9369fe0f475 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:17.258974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d66317-a5d2-4376-b225-a79808bfe61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:17.260064 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d66317-a5d2-4376-b225-a79808bfe61b None admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:17.260586 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d66317-a5d2-4376-b225-a79808bfe61b None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:17.260963 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d66317-a5d2-4376-b225-a79808bfe61b None admin] PATH_INFO: `/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73/users/b92a689fcc4842b3aa1d3a5c3f5e47b4/roles/423aa6ac8b05402da7587e83e5c4ce09` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:17.264149 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d66317-a5d2-4376-b225-a79808bfe61b None admin] RBAC: Authorizing `identity:create_grant(project_id=c60b6aa5591d4178ba9eb5fd05b7ee73, user_id=b92a689fcc4842b3aa1d3a5c3f5e47b4, role_id=423aa6ac8b05402da7587e83e5c4ce09)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:17.266276 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d66317-a5d2-4376-b225-a79808bfe61b None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:17.292316 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:17.299555 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96db62-4a58-4888-801f-4b2ecfb85035 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:17.336164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabe990-ae84-4f28-a9a9-72e055d47d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:17.337064 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabe990-ae84-4f28-a9a9-72e055d47d58 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:17.337168 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabe990-ae84-4f28-a9a9-72e055d47d58 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:17.337439 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabe990-ae84-4f28-a9a9-72e055d47d58 None admin] PATH_INFO: `/v3/users/b92a689fcc4842b3aa1d3a5c3f5e47b4` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:17.338280 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 68/133] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Aug 26 00:34:17 2022] PUT /identity/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73/users/b92a689fcc4842b3aa1d3a5c3f5e47b4/roles/423aa6ac8b05402da7587e83e5c4ce09 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:34:17.338597 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcabe990-ae84-4f28-a9a9-72e055d47d58 None admin] RBAC: Authorizing `identity:get_user(user_id=b92a689fcc4842b3aa1d3a5c3f5e47b4)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:17.339925 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d66317-a5d2-4376-b225-a79808bfe61b None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:17.340246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcabe990-ae84-4f28-a9a9-72e055d47d58 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:17.346884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 66/134] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:17 2022] GET /identity/v3/users/b92a689fcc4842b3aa1d3a5c3f5e47b4 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:17.352535 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cf86f-57e1-4e6a-96d7-796b51e5a804 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:17.356043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cf86f-57e1-4e6a-96d7-796b51e5a804 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:17.356703 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0cf86f-57e1-4e6a-96d7-796b51e5a804 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:17.358384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 67/135] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:17.359330 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0cf86f-57e1-4e6a-96d7-796b51e5a804 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:17.475724 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:17.522385 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d66317-a5d2-4376-b225-a79808bfe61b None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:17.559400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0cf86f-57e1-4e6a-96d7-796b51e5a804 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:17.566225 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a752ee-8e7b-41ff-87ab-18f0a0dfffd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:17.566858 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a752ee-8e7b-41ff-87ab-18f0a0dfffd2 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:17.567142 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a752ee-8e7b-41ff-87ab-18f0a0dfffd2 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:17.567419 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a752ee-8e7b-41ff-87ab-18f0a0dfffd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:17.569814 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a752ee-8e7b-41ff-87ab-18f0a0dfffd2 None admin] RBAC: Adding query filter params (role.id=b68669d0f20b4ac8a1a7d46142b03424, scope.project.id=c60b6aa5591d4178ba9eb5fd05b7ee73, user.id=f52fa6565d874f4e9d40b577d033f854) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:17.570138 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a752ee-8e7b-41ff-87ab-18f0a0dfffd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:17.571728 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a752ee-8e7b-41ff-87ab-18f0a0dfffd2 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:17.578055 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 69/136] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:34:17 2022] GET /identity/v3/role_assignments?user.id=f52fa6565d874f4e9d40b577d033f854&scope.project.id=c60b6aa5591d4178ba9eb5fd05b7ee73&role.id=b68669d0f20b4ac8a1a7d46142b03424 => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:17.579559 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6d592-d0ea-4038-ac67-206771a2cc77 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:17.580216 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6d592-d0ea-4038-ac67-206771a2cc77 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:17.580944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6d592-d0ea-4038-ac67-206771a2cc77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:17.601269 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-5ee6d592-d0ea-4038-ac67-206771a2cc77 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:17.602842 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-5ee6d592-d0ea-4038-ac67-206771a2cc77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:17.605802 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc047bd5-f10d-4272-aabf-4bd7f8b0b621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:17.607866 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc047bd5-f10d-4272-aabf-4bd7f8b0b621 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:17.608351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc047bd5-f10d-4272-aabf-4bd7f8b0b621 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:17.608847 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc047bd5-f10d-4272-aabf-4bd7f8b0b621 None admin] PATH_INFO: `/v3/projects/d7c37d24ebc147969c92537604aabc6b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:17.610218 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc047bd5-f10d-4272-aabf-4bd7f8b0b621 None admin] RBAC: Authorizing `identity:get_project(project_id=d7c37d24ebc147969c92537604aabc6b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:17.612663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc047bd5-f10d-4272-aabf-4bd7f8b0b621 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:17.615600 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 68/137] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:17 2022] GET /identity/v3/projects/d7c37d24ebc147969c92537604aabc6b => generated 312 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:17.621074 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc047bd5-f10d-4272-aabf-4bd7f8b0b621 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:17.661534 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:17.746848 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc047bd5-f10d-4272-aabf-4bd7f8b0b621 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:17.759174 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 70/138] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:17.806701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b3b46-af78-43d1-a84b-cafa4cd90780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:17.807759 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b3b46-af78-43d1-a84b-cafa4cd90780 None admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:17.808235 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b3b46-af78-43d1-a84b-cafa4cd90780 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:17.808404 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03b1d2-c243-46a1-badb-4d28ebda9fbd None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:17.808733 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03b1d2-c243-46a1-badb-4d28ebda9fbd None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:17.809016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03b1d2-c243-46a1-badb-4d28ebda9fbd None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:17.809341 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b3b46-af78-43d1-a84b-cafa4cd90780 None admin] PATH_INFO: `/v3/projects/d7c37d24ebc147969c92537604aabc6b/users/b92a689fcc4842b3aa1d3a5c3f5e47b4/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:17.816495 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 71/139] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:34:17.816495 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0b775-be6f-43e3-be3a-8e4cc846639c None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:17.816495 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0b775-be6f-43e3-be3a-8e4cc846639c None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:17.816888 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0b775-be6f-43e3-be3a-8e4cc846639c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:17.818467 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b3b46-af78-43d1-a84b-cafa4cd90780 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7c37d24ebc147969c92537604aabc6b, user_id=b92a689fcc4842b3aa1d3a5c3f5e47b4, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:17.820514 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b3b46-af78-43d1-a84b-cafa4cd90780 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:17.837197 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-50e0b775-be6f-43e3-be3a-8e4cc846639c None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:17.838555 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-50e0b775-be6f-43e3-be3a-8e4cc846639c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:17.893908 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:17.912132 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 72/140] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:34:17.922913 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 69/141] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Aug 26 00:34:17 2022] PUT /identity/v3/projects/d7c37d24ebc147969c92537604aabc6b/users/b92a689fcc4842b3aa1d3a5c3f5e47b4/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:34:18.094774 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0b775-be6f-43e3-be3a-8e4cc846639c None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:18.219667 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0b775-be6f-43e3-be3a-8e4cc846639c None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:18.259924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d9b0f-8f2b-4e0a-8eb9-312b55169a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:18.260947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d9b0f-8f2b-4e0a-8eb9-312b55169a79 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:18.261437 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d9b0f-8f2b-4e0a-8eb9-312b55169a79 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:18.261900 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d9b0f-8f2b-4e0a-8eb9-312b55169a79 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:18.272657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d9b0f-8f2b-4e0a-8eb9-312b55169a79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:18.277770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-1d4d9b0f-8f2b-4e0a-8eb9-312b55169a79 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:18.277770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:18.277770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:18.277770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:18.277770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:18.277770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:18.277770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:18.277770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:18.277770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:18.277770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:18.277770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:18.277770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:18.277770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:18.277770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:18.277770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:18.277770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:18.278971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:18.278971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:18.278971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:18.278971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:18.278971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:18.278971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:18.278971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:18.278971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:18.278971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:18.278971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:18.278971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:18.278971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:18.278971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:18.278971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:18.278971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:18.278971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:18.279738 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:18.279738 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:18.279738 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:18.279738 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:18.279738 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:18.279738 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:18.279738 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:18.279738 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:18.279738 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:18.279738 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:18.279738 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:18.279738 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:34:18.281754 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 73/142] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:18.282017 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b3b46-af78-43d1-a84b-cafa4cd90780 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:18.288912 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d9b0f-8f2b-4e0a-8eb9-312b55169a79 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:18.336826 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:18.388951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:18.392550 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d9b0f-8f2b-4e0a-8eb9-312b55169a79 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:18.398987 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b3b46-af78-43d1-a84b-cafa4cd90780 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:18.440656 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdd6ce-cf93-48df-aae0-0dd3ec3df58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:18.441281 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdd6ce-cf93-48df-aae0-0dd3ec3df58e None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:18.441564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdd6ce-cf93-48df-aae0-0dd3ec3df58e None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:18.441836 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdd6ce-cf93-48df-aae0-0dd3ec3df58e None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:18.442519 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdd6ce-cf93-48df-aae0-0dd3ec3df58e None admin] RBAC: Adding query filter params (name=member) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:18.442823 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdd6ce-cf93-48df-aae0-0dd3ec3df58e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:18.444223 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdd6ce-cf93-48df-aae0-0dd3ec3df58e None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:18.448696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343e582-7bea-430c-9b6e-9efcb3774ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:18.449463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343e582-7bea-430c-9b6e-9efcb3774ed9 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:18.449900 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343e582-7bea-430c-9b6e-9efcb3774ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:18.450307 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343e582-7bea-430c-9b6e-9efcb3774ed9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:18.455104 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c343e582-7bea-430c-9b6e-9efcb3774ed9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:18.457480 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c343e582-7bea-430c-9b6e-9efcb3774ed9 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:18.459309 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 70/143] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:34:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:18.461667 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 74/144] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:34:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:18.464692 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343e582-7bea-430c-9b6e-9efcb3774ed9 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:18.469838 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdd6ce-cf93-48df-aae0-0dd3ec3df58e None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:18.648198 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343e582-7bea-430c-9b6e-9efcb3774ed9 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:18.684623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea261ad-00f1-4d74-b3ea-63400bedfb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:18.685312 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea261ad-00f1-4d74-b3ea-63400bedfb01 None admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:18.685591 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea261ad-00f1-4d74-b3ea-63400bedfb01 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:18.685868 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea261ad-00f1-4d74-b3ea-63400bedfb01 None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:18.686520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea261ad-00f1-4d74-b3ea-63400bedfb01 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:18.687655 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdd6ce-cf93-48df-aae0-0dd3ec3df58e None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:18.688091 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea261ad-00f1-4d74-b3ea-63400bedfb01 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:18.695851 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:18.736950 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dad265-6444-4877-af71-1b838e68ae7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:18.737600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dad265-6444-4877-af71-1b838e68ae7b None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:18.737885 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dad265-6444-4877-af71-1b838e68ae7b None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:18.738167 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dad265-6444-4877-af71-1b838e68ae7b None admin] PATH_INFO: `/v3/users/b92a689fcc4842b3aa1d3a5c3f5e47b4` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:18.745161 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dad265-6444-4877-af71-1b838e68ae7b None admin] RBAC: Authorizing `identity:get_user(user_id=b92a689fcc4842b3aa1d3a5c3f5e47b4)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:18.746700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dad265-6444-4877-af71-1b838e68ae7b None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:18.748703 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 75/145] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:18 2022] GET /identity/v3/users/b92a689fcc4842b3aa1d3a5c3f5e47b4 => generated 275 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:18.753717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dad265-6444-4877-af71-1b838e68ae7b None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:18.802091 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 71/146] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Aug 26 00:34:18 2022] POST /identity/v3/users => generated 283 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:34:18.862081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dad265-6444-4877-af71-1b838e68ae7b None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:18.913400 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b091880-4995-43d3-be31-026e80488a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:18.913949 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b091880-4995-43d3-be31-026e80488a38 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:18.914232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b091880-4995-43d3-be31-026e80488a38 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:18.914507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b091880-4995-43d3-be31-026e80488a38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:18.972627 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:19.006153 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b091880-4995-43d3-be31-026e80488a38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:19.007629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b091880-4995-43d3-be31-026e80488a38 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:19.132899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 76/147] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:34:19.138614 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea261ad-00f1-4d74-b3ea-63400bedfb01 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:19.156823 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:19.244531 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea261ad-00f1-4d74-b3ea-63400bedfb01 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:19.278210 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159509b-6987-402d-b6a3-b1f634340fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:19.279138 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159509b-6987-402d-b6a3-b1f634340fe1 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:19.279418 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159509b-6987-402d-b6a3-b1f634340fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:19.279694 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159509b-6987-402d-b6a3-b1f634340fe1 None admin] PATH_INFO: `/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:19.280860 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159509b-6987-402d-b6a3-b1f634340fe1 None admin] RBAC: Authorizing `identity:get_project(project_id=c60b6aa5591d4178ba9eb5fd05b7ee73)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:19.282310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159509b-6987-402d-b6a3-b1f634340fe1 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:19.284447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 72/148] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:19 2022] GET /identity/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:19.290297 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b091880-4995-43d3-be31-026e80488a38 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:19.369434 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:19.399917 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b091880-4995-43d3-be31-026e80488a38 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:19.452168 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f4812-008d-473a-926a-68ffa0c09471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:19.452808 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f4812-008d-473a-926a-68ffa0c09471 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:19.453087 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f4812-008d-473a-926a-68ffa0c09471 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:19.453357 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f4812-008d-473a-926a-68ffa0c09471 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:19.454081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f4812-008d-473a-926a-68ffa0c09471 None admin] RBAC: Adding query filter params (role.id=5032636d8729461c9926a59b9bcd041b, scope.project.id=c60b6aa5591d4178ba9eb5fd05b7ee73, user.id=b92a689fcc4842b3aa1d3a5c3f5e47b4) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:19.454468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f4812-008d-473a-926a-68ffa0c09471 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:19.456441 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f4812-008d-473a-926a-68ffa0c09471 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:19.462866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 77/149] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:34:19 2022] GET /identity/v3/role_assignments?user.id=b92a689fcc4842b3aa1d3a5c3f5e47b4&scope.project.id=c60b6aa5591d4178ba9eb5fd05b7ee73&role.id=5032636d8729461c9926a59b9bcd041b => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:19.934734 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f9fa8-a3ba-42a1-8d10-361e37e98fde None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:19.935350 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f9fa8-a3ba-42a1-8d10-361e37e98fde None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:19.935350 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f9fa8-a3ba-42a1-8d10-361e37e98fde None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:19.936695 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 73/150] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:19.943886 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256a1ef-7b8b-4acf-a00c-afe3100c9204 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:19.944357 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256a1ef-7b8b-4acf-a00c-afe3100c9204 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:19.944438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256a1ef-7b8b-4acf-a00c-afe3100c9204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:19.963451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-d256a1ef-7b8b-4acf-a00c-afe3100c9204 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:19.964666 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-d256a1ef-7b8b-4acf-a00c-afe3100c9204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:20.046492 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 78/151] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:20.161992 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19217e4b-f730-4a20-a07b-e8979076935e None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:20.162320 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19217e4b-f730-4a20-a07b-e8979076935e None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:20.162554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19217e4b-f730-4a20-a07b-e8979076935e None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:20.163899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 74/152] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:20.178461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0dac82-4657-4846-80a3-4d7d591f8247 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:20.178738 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0dac82-4657-4846-80a3-4d7d591f8247 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:20.179014 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0dac82-4657-4846-80a3-4d7d591f8247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:20.205599 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-7e0dac82-4657-4846-80a3-4d7d591f8247 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:20.206716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-7e0dac82-4657-4846-80a3-4d7d591f8247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:20.280952 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:20.287474 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 79/153] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:20.378673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746ac75-abb3-4a35-8e21-73dc9836f1a7 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:20.378998 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746ac75-abb3-4a35-8e21-73dc9836f1a7 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:20.379228 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746ac75-abb3-4a35-8e21-73dc9836f1a7 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:20.380592 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 75/154] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:20.385980 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caefaf7-1e84-4a88-88e6-6f9bd2ef2a71 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:20.386301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caefaf7-1e84-4a88-88e6-6f9bd2ef2a71 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:20.386532 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caefaf7-1e84-4a88-88e6-6f9bd2ef2a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:20.413679 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-6caefaf7-1e84-4a88-88e6-6f9bd2ef2a71 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:20.414866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-6caefaf7-1e84-4a88-88e6-6f9bd2ef2a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:20.499477 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 80/155] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:20.567750 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0746ac75-abb3-4a35-8e21-73dc9836f1a7 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:20.671991 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:20.703754 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0746ac75-abb3-4a35-8e21-73dc9836f1a7 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:20.716984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caefaf7-1e84-4a88-88e6-6f9bd2ef2a71 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:20.741060 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb86ae-2d73-429d-a5ea-65481c3be38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:20.741703 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb86ae-2d73-429d-a5ea-65481c3be38a None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:20.741987 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb86ae-2d73-429d-a5ea-65481c3be38a None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:20.742263 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb86ae-2d73-429d-a5ea-65481c3be38a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:20.753323 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb86ae-2d73-429d-a5ea-65481c3be38a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:20.758846 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-5fcb86ae-2d73-429d-a5ea-65481c3be38a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:34:20.758846 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:20.758846 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:20.758846 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:20.758846 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:20.758846 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:20.758846 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:20.758846 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:20.758846 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:20.758846 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:20.758846 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:20.758846 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:20.758846 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:20.758846 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:20.758846 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:20.758846 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:20.759798 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:20.759798 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:20.759798 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:20.759798 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:20.759798 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:20.759798 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:20.759798 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:20.759798 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:20.759798 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:20.759798 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:20.759798 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:20.759798 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:20.759798 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:20.759798 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:20.759798 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:20.759798 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:20.760660 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:20.760660 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:20.760660 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:20.760660 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:20.760660 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:20.760660 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:20.760660 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:20.760660 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:20.760660 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:20.760660 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:20.760660 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:34:20.760660 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:34:20.760660 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 76/156] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:34:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:20.769610 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb86ae-2d73-429d-a5ea-65481c3be38a None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:20.914714 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb86ae-2d73-429d-a5ea-65481c3be38a None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:20.928196 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caefaf7-1e84-4a88-88e6-6f9bd2ef2a71 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:20.985333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dde454-0b0d-45fb-8d74-b38dd3fbbf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:20.986892 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dde454-0b0d-45fb-8d74-b38dd3fbbf8d None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:20.987239 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dde454-0b0d-45fb-8d74-b38dd3fbbf8d None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:20.987652 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dde454-0b0d-45fb-8d74-b38dd3fbbf8d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:20.991035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:21.007665 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe18add-1a39-4392-83e5-74fa022c67f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:21.008303 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe18add-1a39-4392-83e5-74fa022c67f9 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:21.008585 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe18add-1a39-4392-83e5-74fa022c67f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:21.009020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe18add-1a39-4392-83e5-74fa022c67f9 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:21.009728 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe18add-1a39-4392-83e5-74fa022c67f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:21.010193 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe18add-1a39-4392-83e5-74fa022c67f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:21.011470 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe18add-1a39-4392-83e5-74fa022c67f9 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:21.021639 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dde454-0b0d-45fb-8d74-b38dd3fbbf8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:21.026443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-65dde454-0b0d-45fb-8d74-b38dd3fbbf8d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 00:34:21.026443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:21.026443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:21.026443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:21.026443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:21.026443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:21.026443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:21.026443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:21.026443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:21.026443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:21.026443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:21.026443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:21.026443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:21.026443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:21.026443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:21.026443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:21.027384 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:21.027384 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:21.027384 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:21.027384 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:21.027384 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:21.027384 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:21.027384 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:21.027384 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:21.027384 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:21.027384 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:21.027384 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:21.027384 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:21.027384 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:21.027384 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:21.027384 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:21.027384 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:21.028859 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:21.028859 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:21.028859 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:21.028859 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:21.028859 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:21.028859 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:21.028859 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:21.028859 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:21.028859 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:21.028859 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:21.028859 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 00:34:21.028859 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:34:21.028859 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 81/157] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Aug 26 00:34:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:21.034475 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 77/158] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:21.040139 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dde454-0b0d-45fb-8d74-b38dd3fbbf8d None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:21.040321 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe18add-1a39-4392-83e5-74fa022c67f9 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:21.049400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:21.202165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe18add-1a39-4392-83e5-74fa022c67f9 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:21.210509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dde454-0b0d-45fb-8d74-b38dd3fbbf8d None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:21.246974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e09b6b-2ea5-41c7-abdb-5bc40b72985d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:21.249410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e09b6b-2ea5-41c7-abdb-5bc40b72985d None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:21.249908 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e09b6b-2ea5-41c7-abdb-5bc40b72985d None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:21.251156 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e09b6b-2ea5-41c7-abdb-5bc40b72985d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:21.266620 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e09b6b-2ea5-41c7-abdb-5bc40b72985d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:21.271757 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34e082-284a-4ce5-99a5-b495e53dd8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:21.272310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-73e09b6b-2ea5-41c7-abdb-5bc40b72985d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 00:34:21.272310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:21.272310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:21.272310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:21.272310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:21.272310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:21.272310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:21.272310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:21.272310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:21.272310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:21.272310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:21.272310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:21.272310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:21.272310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:21.272310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:21.272310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:21.273257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:21.273257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:21.273257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:21.273257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:21.273257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:21.273257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:21.273257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:21.273257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:21.273257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:21.273257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:21.273257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:21.273257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:21.273257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:21.273257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:21.273257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:21.273257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:21.274048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:21.274048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:21.274048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:21.274048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:21.274048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:21.274048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:21.274048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:21.274048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:21.274048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:21.274048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:21.274048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 00:34:21.274048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:34:21.274645 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34e082-284a-4ce5-99a5-b495e53dd8e2 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:21.274645 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34e082-284a-4ce5-99a5-b495e53dd8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:21.274645 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34e082-284a-4ce5-99a5-b495e53dd8e2 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:21.274645 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34e082-284a-4ce5-99a5-b495e53dd8e2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:21.274645 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34e082-284a-4ce5-99a5-b495e53dd8e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:21.275234 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34e082-284a-4ce5-99a5-b495e53dd8e2 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:21.276375 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 82/159] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:21.281523 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e09b6b-2ea5-41c7-abdb-5bc40b72985d None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:21.295210 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 78/160] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:34:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:21.295210 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887bf94-5393-412e-ba9a-17d737def372 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:21.295210 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887bf94-5393-412e-ba9a-17d737def372 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:21.295210 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887bf94-5393-412e-ba9a-17d737def372 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:21.295210 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 79/161] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:21.295210 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887bf94-5393-412e-ba9a-17d737def372 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:21.382677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:21.432831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:21.488687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e09b6b-2ea5-41c7-abdb-5bc40b72985d None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:21.524510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e9e81-5803-407b-a6b6-a619170504c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:21.525152 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887bf94-5393-412e-ba9a-17d737def372 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:21.525272 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e9e81-5803-407b-a6b6-a619170504c6 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:21.525528 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e9e81-5803-407b-a6b6-a619170504c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:21.525874 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e9e81-5803-407b-a6b6-a619170504c6 None admin] PATH_INFO: `/v3/users/795669ccac0f433b900784c76b5b0142` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:21.527259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e9e81-5803-407b-a6b6-a619170504c6 None admin] RBAC: Authorizing `identity:get_user(user_id=795669ccac0f433b900784c76b5b0142)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:21.529177 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e9e81-5803-407b-a6b6-a619170504c6 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:21.530848 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 83/162] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:21 2022] GET /identity/v3/users/795669ccac0f433b900784c76b5b0142 => generated 247 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:21.531701 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e9e81-5803-407b-a6b6-a619170504c6 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:21.591694 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d069cc-695f-4ea2-b221-e2a3f89c2850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:21.592318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d069cc-695f-4ea2-b221-e2a3f89c2850 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:21.596307 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d069cc-695f-4ea2-b221-e2a3f89c2850 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:21.596670 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d069cc-695f-4ea2-b221-e2a3f89c2850 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:21.597358 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d069cc-695f-4ea2-b221-e2a3f89c2850 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:21.597668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d069cc-695f-4ea2-b221-e2a3f89c2850 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:21.599219 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d069cc-695f-4ea2-b221-e2a3f89c2850 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:21.614726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 80/163] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:34:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:21.615980 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0a7c8-dfc0-4588-bbbd-78286676e208 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:21.616265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0a7c8-dfc0-4588-bbbd-78286676e208 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:21.616561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0a7c8-dfc0-4588-bbbd-78286676e208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:21.651142 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-16b0a7c8-dfc0-4588-bbbd-78286676e208 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:21.652334 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-16b0a7c8-dfc0-4588-bbbd-78286676e208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:21.709006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e9e81-5803-407b-a6b6-a619170504c6 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:21.743324 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:21.777531 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81d1e6-b404-4a6a-bc18-584568b467e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:21.780536 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81d1e6-b404-4a6a-bc18-584568b467e0 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:21.781261 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81d1e6-b404-4a6a-bc18-584568b467e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:21.785626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81d1e6-b404-4a6a-bc18-584568b467e0 None admin] PATH_INFO: `/v3/users/b92a689fcc4842b3aa1d3a5c3f5e47b4` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:21.787199 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81d1e6-b404-4a6a-bc18-584568b467e0 None admin] RBAC: Authorizing `identity:get_user(user_id=b92a689fcc4842b3aa1d3a5c3f5e47b4)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:21.790024 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 81/164] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:21.791452 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81d1e6-b404-4a6a-bc18-584568b467e0 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:21.791572 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0a7c8-dfc0-4588-bbbd-78286676e208 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:21.802342 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 84/165] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:21 2022] GET /identity/v3/users/b92a689fcc4842b3aa1d3a5c3f5e47b4 => generated 275 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:21.803160 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81d1e6-b404-4a6a-bc18-584568b467e0 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:21.932836 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:22.019830 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0a7c8-dfc0-4588-bbbd-78286676e208 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:22.042681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81d1e6-b404-4a6a-bc18-584568b467e0 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:22.110530 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7430bf-0a70-4c02-bfaa-d223f2178b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:22.111584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7430bf-0a70-4c02-bfaa-d223f2178b7e None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:22.112093 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7430bf-0a70-4c02-bfaa-d223f2178b7e None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:22.112594 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7430bf-0a70-4c02-bfaa-d223f2178b7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:22.134193 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041af79-addc-4ae8-8b3f-b0bf077e0d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:22.135227 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041af79-addc-4ae8-8b3f-b0bf077e0d0b None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:22.135510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041af79-addc-4ae8-8b3f-b0bf077e0d0b None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:22.135776 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041af79-addc-4ae8-8b3f-b0bf077e0d0b None admin] PATH_INFO: `/v3/users/f52fa6565d874f4e9d40b577d033f854` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:22.137006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041af79-addc-4ae8-8b3f-b0bf077e0d0b None admin] RBAC: Authorizing `identity:get_user(user_id=f52fa6565d874f4e9d40b577d033f854)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:22.138527 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041af79-addc-4ae8-8b3f-b0bf077e0d0b None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:22.140381 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 85/166] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:21 2022] GET /identity/v3/users/f52fa6565d874f4e9d40b577d033f854 => generated 289 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:22.140989 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041af79-addc-4ae8-8b3f-b0bf077e0d0b None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:22.231075 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:22.293763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7430bf-0a70-4c02-bfaa-d223f2178b7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:22.295577 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7430bf-0a70-4c02-bfaa-d223f2178b7e None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:22.301457 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041af79-addc-4ae8-8b3f-b0bf077e0d0b None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:22.350118 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b72a5-d62e-45b4-b11e-62aea607b31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:22.350757 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b72a5-d62e-45b4-b11e-62aea607b31b None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:22.351027 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b72a5-d62e-45b4-b11e-62aea607b31b None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:22.351297 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b72a5-d62e-45b4-b11e-62aea607b31b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:22.359731 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:22.413461 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 82/167] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 623 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:34:22.414649 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7430bf-0a70-4c02-bfaa-d223f2178b7e None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:22.455070 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394b72a5-d62e-45b4-b11e-62aea607b31b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:22.456618 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394b72a5-d62e-45b4-b11e-62aea607b31b None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:22.526330 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:22.550261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7430bf-0a70-4c02-bfaa-d223f2178b7e None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:22.587788 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 86/168] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:34:22.588096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5dfd6f-df58-4dc3-a230-8df84889689a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:22.588702 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dfd6f-df58-4dc3-a230-8df84889689a None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:22.588987 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dfd6f-df58-4dc3-a230-8df84889689a None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:22.589264 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dfd6f-df58-4dc3-a230-8df84889689a None admin] PATH_INFO: `/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:22.590214 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b72a5-d62e-45b4-b11e-62aea607b31b None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:22.590508 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5dfd6f-df58-4dc3-a230-8df84889689a None admin] RBAC: Authorizing `identity:get_project(project_id=c60b6aa5591d4178ba9eb5fd05b7ee73)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:22.591979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5dfd6f-df58-4dc3-a230-8df84889689a None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:22.594122 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 83/169] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:22 2022] GET /identity/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:22.600589 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f43ec-065a-422d-b938-c5ead1ce059b None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:22.600589 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f43ec-065a-422d-b938-c5ead1ce059b None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:22.600589 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f43ec-065a-422d-b938-c5ead1ce059b None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:22.600589 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 84/170] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:22.600589 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f43ec-065a-422d-b938-c5ead1ce059b None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:22.628994 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:22.705531 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b72a5-d62e-45b4-b11e-62aea607b31b None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:22.710483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f43ec-065a-422d-b938-c5ead1ce059b None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:22.741902 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a07d92-e969-4ddf-8d1f-d381b784f12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:22.742926 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a07d92-e969-4ddf-8d1f-d381b784f12b None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:22.743409 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a07d92-e969-4ddf-8d1f-d381b784f12b None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:22.743992 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a07d92-e969-4ddf-8d1f-d381b784f12b None admin] PATH_INFO: `/v3/roles/member` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:22.746965 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457fd84b-f8d6-4af1-ba35-dc0e3ef62f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:22.747708 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fd84b-f8d6-4af1-ba35-dc0e3ef62f9b None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:22.748007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fd84b-f8d6-4af1-ba35-dc0e3ef62f9b None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:22.748268 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fd84b-f8d6-4af1-ba35-dc0e3ef62f9b None admin] PATH_INFO: `/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:22.749433 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457fd84b-f8d6-4af1-ba35-dc0e3ef62f9b None admin] RBAC: Authorizing `identity:get_project(project_id=c60b6aa5591d4178ba9eb5fd05b7ee73)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:22.750896 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457fd84b-f8d6-4af1-ba35-dc0e3ef62f9b None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:22.753100 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 85/171] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:22 2022] GET /identity/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:22.753877 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457fd84b-f8d6-4af1-ba35-dc0e3ef62f9b None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:22.758395 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a07d92-e969-4ddf-8d1f-d381b784f12b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:22.763616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-a7a07d92-e969-4ddf-8d1f-d381b784f12b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:22.763616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:22.763616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:22.763616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:22.763616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:22.763616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:22.763616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:22.763616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:22.763616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:22.763616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:22.763616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:22.763616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:22.763616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:22.763616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:22.763616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:22.763616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:22.764541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:22.764541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:22.764541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:22.764541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:22.764541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:22.764541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:22.764541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:22.764541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:22.764541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:22.764541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:22.764541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:22.764541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:22.764541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:22.764541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:22.764541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:22.764541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:22.765436 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:22.765436 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:22.765436 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:22.765436 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:22.765436 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:22.765436 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:22.765436 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:22.765436 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:22.765436 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:22.765436 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:22.765436 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:22.765436 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:34:22.768375 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 87/172] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:22.769408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a07d92-e969-4ddf-8d1f-d381b784f12b None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:22.786766 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:22.863022 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:22.869976 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457fd84b-f8d6-4af1-ba35-dc0e3ef62f9b None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:22.880051 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a07d92-e969-4ddf-8d1f-d381b784f12b None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:22.907935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34449308-6f83-4c11-940c-3c23a3e2a299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:22.908638 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34449308-6f83-4c11-940c-3c23a3e2a299 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:22.908921 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34449308-6f83-4c11-940c-3c23a3e2a299 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:22.909197 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34449308-6f83-4c11-940c-3c23a3e2a299 None admin] PATH_INFO: `/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:22.910328 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34449308-6f83-4c11-940c-3c23a3e2a299 None admin] RBAC: Authorizing `identity:get_project(project_id=c60b6aa5591d4178ba9eb5fd05b7ee73)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:22.911789 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34449308-6f83-4c11-940c-3c23a3e2a299 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:22.913442 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 86/173] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:22 2022] GET /identity/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:22.914630 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a79a4d-941a-4c81-a2eb-8fb1b73ba236 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:22.914915 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a79a4d-941a-4c81-a2eb-8fb1b73ba236 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:22.915191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a79a4d-941a-4c81-a2eb-8fb1b73ba236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:22.922666 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7b892-9d22-49ce-882c-f499f2d74ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:22.923344 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7b892-9d22-49ce-882c-f499f2d74ad2 None admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:22.923658 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7b892-9d22-49ce-882c-f499f2d74ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:22.923939 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7b892-9d22-49ce-882c-f499f2d74ad2 None admin] PATH_INFO: `/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73/users/f52fa6565d874f4e9d40b577d033f854/roles/b68669d0f20b4ac8a1a7d46142b03424` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:22.925947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a7b892-9d22-49ce-882c-f499f2d74ad2 None admin] RBAC: Authorizing `identity:create_grant(project_id=c60b6aa5591d4178ba9eb5fd05b7ee73, user_id=f52fa6565d874f4e9d40b577d033f854, role_id=b68669d0f20b4ac8a1a7d46142b03424)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:22.927388 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a7b892-9d22-49ce-882c-f499f2d74ad2 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:22.934140 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-d6a79a4d-941a-4c81-a2eb-8fb1b73ba236 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:22.935329 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-d6a79a4d-941a-4c81-a2eb-8fb1b73ba236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:23.000592 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 87/174] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:23.001418 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a79a4d-941a-4c81-a2eb-8fb1b73ba236 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:23.016339 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 88/175] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Aug 26 00:34:22 2022] PUT /identity/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73/users/f52fa6565d874f4e9d40b577d033f854/roles/b68669d0f20b4ac8a1a7d46142b03424 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:34:23.029884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7b892-9d22-49ce-882c-f499f2d74ad2 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:23.108238 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:23.118436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a79a4d-941a-4c81-a2eb-8fb1b73ba236 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:23.142141 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:23.158364 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e6127-d6ce-4806-94b7-010908dabe2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:23.158986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e6127-d6ce-4806-94b7-010908dabe2e None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:23.159261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e6127-d6ce-4806-94b7-010908dabe2e None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:23.159534 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e6127-d6ce-4806-94b7-010908dabe2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:23.161100 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e6127-d6ce-4806-94b7-010908dabe2e None admin] RBAC: Adding query filter params (role.id=1a33e3c8cb8347feb9498595886cc621, scope.project.id=c60b6aa5591d4178ba9eb5fd05b7ee73, user.id=795669ccac0f433b900784c76b5b0142) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:23.161413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e6127-d6ce-4806-94b7-010908dabe2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:23.162933 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e6127-d6ce-4806-94b7-010908dabe2e None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:23.171344 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 88/176] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:34:23 2022] GET /identity/v3/role_assignments?user.id=795669ccac0f433b900784c76b5b0142&scope.project.id=c60b6aa5591d4178ba9eb5fd05b7ee73&role.id=1a33e3c8cb8347feb9498595886cc621 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:23.175100 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08779d-5b2f-48f2-a312-e4e1a903ca19 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:23.175100 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08779d-5b2f-48f2-a312-e4e1a903ca19 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:23.175100 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08779d-5b2f-48f2-a312-e4e1a903ca19 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:23.175100 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 89/177] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:23.175523 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca08779d-5b2f-48f2-a312-e4e1a903ca19 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:23.192148 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7b892-9d22-49ce-882c-f499f2d74ad2 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:23.240507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f07f59-976a-4ac5-a6f8-12de4c8b712a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:23.241160 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f07f59-976a-4ac5-a6f8-12de4c8b712a None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:23.241447 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f07f59-976a-4ac5-a6f8-12de4c8b712a None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:23.241716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f07f59-976a-4ac5-a6f8-12de4c8b712a None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:23.242515 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f07f59-976a-4ac5-a6f8-12de4c8b712a None admin] RBAC: Adding query filter params (name=member) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:23.242853 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f07f59-976a-4ac5-a6f8-12de4c8b712a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:23.244248 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f07f59-976a-4ac5-a6f8-12de4c8b712a None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:23.254146 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 89/178] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:34:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:23.255414 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9bc5d-7672-4039-acfc-06f63e75e107 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:23.255697 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9bc5d-7672-4039-acfc-06f63e75e107 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:23.255968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9bc5d-7672-4039-acfc-06f63e75e107 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:23.262224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 90/179] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:23.263529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe22119-55bf-44a1-a74c-ef1cf20d2f39 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:23.263789 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe22119-55bf-44a1-a74c-ef1cf20d2f39 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:23.264118 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe22119-55bf-44a1-a74c-ef1cf20d2f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:23.283368 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-dbe22119-55bf-44a1-a74c-ef1cf20d2f39 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:23.284625 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-dbe22119-55bf-44a1-a74c-ef1cf20d2f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:23.360270 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca08779d-5b2f-48f2-a312-e4e1a903ca19 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:23.369647 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 91/180] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:23.370765 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe22119-55bf-44a1-a74c-ef1cf20d2f39 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:23.413067 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9bad3-b526-428b-969d-b0eb406bce7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:23.413700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9bad3-b526-428b-969d-b0eb406bce7f None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:23.413980 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9bad3-b526-428b-969d-b0eb406bce7f None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:23.414252 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9bad3-b526-428b-969d-b0eb406bce7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:23.414996 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9bad3-b526-428b-969d-b0eb406bce7f None admin] RBAC: Adding query filter params (role.id=423aa6ac8b05402da7587e83e5c4ce09, scope.project.id=c60b6aa5591d4178ba9eb5fd05b7ee73, user.id=b92a689fcc4842b3aa1d3a5c3f5e47b4) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:23.415305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9bad3-b526-428b-969d-b0eb406bce7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:23.416781 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9bad3-b526-428b-969d-b0eb406bce7f None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:23.427322 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 90/181] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:34:23 2022] GET /identity/v3/role_assignments?user.id=b92a689fcc4842b3aa1d3a5c3f5e47b4&scope.project.id=c60b6aa5591d4178ba9eb5fd05b7ee73&role.id=423aa6ac8b05402da7587e83e5c4ce09 => generated 608 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:23.429042 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c111ba7-4319-4376-8482-422c6833d556 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:23.431366 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c111ba7-4319-4376-8482-422c6833d556 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:23.431917 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c111ba7-4319-4376-8482-422c6833d556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:23.465310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-5c111ba7-4319-4376-8482-422c6833d556 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:23.468193 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-5c111ba7-4319-4376-8482-422c6833d556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:23.471719 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:23.505446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:23.577788 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 91/182] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:23.579143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c111ba7-4319-4376-8482-422c6833d556 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:23.581040 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe22119-55bf-44a1-a74c-ef1cf20d2f39 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:23.669856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa11ec6-f362-4e2f-bf63-3328f79651a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:23.670431 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa11ec6-f362-4e2f-bf63-3328f79651a0 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:23.670744 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa11ec6-f362-4e2f-bf63-3328f79651a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:23.671339 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa11ec6-f362-4e2f-bf63-3328f79651a0 None admin] PATH_INFO: `/v3/users/b92a689fcc4842b3aa1d3a5c3f5e47b4` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:23.672306 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa11ec6-f362-4e2f-bf63-3328f79651a0 None admin] RBAC: Authorizing `identity:get_user(user_id=b92a689fcc4842b3aa1d3a5c3f5e47b4)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:23.674456 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa11ec6-f362-4e2f-bf63-3328f79651a0 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:23.676075 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 92/183] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:23 2022] GET /identity/v3/users/b92a689fcc4842b3aa1d3a5c3f5e47b4 => generated 275 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:23.682047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa11ec6-f362-4e2f-bf63-3328f79651a0 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:23.699756 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c111ba7-4319-4376-8482-422c6833d556 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:23.739918 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c70e5-4df8-49d4-a376-cc72fa477a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:23.741145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c70e5-4df8-49d4-a376-cc72fa477a21 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:23.741635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c70e5-4df8-49d4-a376-cc72fa477a21 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:23.742283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c70e5-4df8-49d4-a376-cc72fa477a21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:23.755394 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c70e5-4df8-49d4-a376-cc72fa477a21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:23.760280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-7e4c70e5-4df8-49d4-a376-cc72fa477a21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:34:23.760280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:23.760280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:23.760280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:23.760280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:23.760280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:23.760280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:23.760280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:23.760280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:23.760280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:23.760280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:23.760280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:23.760280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:23.760280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:23.760280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:23.760280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:23.761275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:23.761275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:23.761275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:23.761275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:23.761275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:23.761275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:23.761275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:23.761275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:23.761275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:23.761275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:23.761275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:23.761275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:23.761275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:23.761275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:23.761275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:23.761275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:23.762275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:23.762275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:23.762275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:23.762275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:23.762275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:23.762275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:23.762275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:23.762275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:23.762275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:23.762275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:23.762275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:34:23.762275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:34:23.762275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 92/184] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:34:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:23.762934 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c70e5-4df8-49d4-a376-cc72fa477a21 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:23.775872 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:23.842728 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa11ec6-f362-4e2f-bf63-3328f79651a0 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:23.879998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d0838-f0fe-4cd0-8c3e-67e1bbaae3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:23.880636 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d0838-f0fe-4cd0-8c3e-67e1bbaae3d0 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:23.880917 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d0838-f0fe-4cd0-8c3e-67e1bbaae3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:23.881189 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d0838-f0fe-4cd0-8c3e-67e1bbaae3d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:23.882047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:23.893997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c70e5-4df8-49d4-a376-cc72fa477a21 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:23.939096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f6207-9d84-4d45-b0ab-8dbf25b4f0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:23.940592 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f6207-9d84-4d45-b0ab-8dbf25b4f0d4 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:23.941251 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f6207-9d84-4d45-b0ab-8dbf25b4f0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:23.941542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f6207-9d84-4d45-b0ab-8dbf25b4f0d4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:23.955134 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f6207-9d84-4d45-b0ab-8dbf25b4f0d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:23.961061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-920f6207-9d84-4d45-b0ab-8dbf25b4f0d4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 00:34:23.961061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:23.961061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:23.961061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:23.961061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:23.961061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:23.961061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:23.961061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:23.961061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:23.961061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:23.961061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:23.961061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:23.961061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:23.961061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:23.961061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:23.961061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:23.963011 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:23.963011 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:23.963011 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:23.963011 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:23.963011 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:23.963011 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:23.963011 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:23.963011 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:23.963011 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:23.963011 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:23.963011 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:23.963011 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:23.963011 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:23.963011 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:23.963011 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:23.963011 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:23.964049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:23.964049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:23.964049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:23.964049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:23.964049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:23.964049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:23.964049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:23.964049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:23.964049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:23.964049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:23.964049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 00:34:23.964049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:34:23.964049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 93/185] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Aug 26 00:34:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:23.964049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f6207-9d84-4d45-b0ab-8dbf25b4f0d4 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:23.968119 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d0838-f0fe-4cd0-8c3e-67e1bbaae3d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:23.969574 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d0838-f0fe-4cd0-8c3e-67e1bbaae3d0 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:24.044043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:24.076969 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 93/186] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:34:24.077817 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d0838-f0fe-4cd0-8c3e-67e1bbaae3d0 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:24.108525 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:24.125431 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f6207-9d84-4d45-b0ab-8dbf25b4f0d4 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:24.162414 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9ce42-6125-4b3d-9654-11fabfa64434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:24.163032 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9ce42-6125-4b3d-9654-11fabfa64434 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:24.163313 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9ce42-6125-4b3d-9654-11fabfa64434 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:24.163593 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9ce42-6125-4b3d-9654-11fabfa64434 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:24.164266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e9ce42-6125-4b3d-9654-11fabfa64434 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:24.164598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e9ce42-6125-4b3d-9654-11fabfa64434 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:24.166059 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e9ce42-6125-4b3d-9654-11fabfa64434 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:24.174462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 94/187] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:24.175103 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9ce42-6125-4b3d-9654-11fabfa64434 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:24.181629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d0838-f0fe-4cd0-8c3e-67e1bbaae3d0 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:24.219652 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3991163-96de-4495-bcc6-6a862d76c958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:24.220256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3991163-96de-4495-bcc6-6a862d76c958 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:24.221484 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3991163-96de-4495-bcc6-6a862d76c958 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:24.221802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3991163-96de-4495-bcc6-6a862d76c958 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:24.223212 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3991163-96de-4495-bcc6-6a862d76c958 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:24.224849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3991163-96de-4495-bcc6-6a862d76c958 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:24.228624 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 94/188] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:34:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:24.228624 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3991163-96de-4495-bcc6-6a862d76c958 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:24.286042 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9ce42-6125-4b3d-9654-11fabfa64434 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:24.318638 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:24.324016 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1fae9-4783-4e12-aa9f-b9c3d614b30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:24.324657 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1fae9-4783-4e12-aa9f-b9c3d614b30f None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:24.324946 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1fae9-4783-4e12-aa9f-b9c3d614b30f None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:24.325220 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1fae9-4783-4e12-aa9f-b9c3d614b30f None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:24.325893 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c1fae9-4783-4e12-aa9f-b9c3d614b30f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:24.326197 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c1fae9-4783-4e12-aa9f-b9c3d614b30f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:24.327591 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c1fae9-4783-4e12-aa9f-b9c3d614b30f None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:24.330343 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:24.333160 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3991163-96de-4495-bcc6-6a862d76c958 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:24.335889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 95/189] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:34:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:24.337334 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1fae9-4783-4e12-aa9f-b9c3d614b30f None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:24.369305 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad312d-4d7a-4ba2-ba29-61eff748e5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:24.370100 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad312d-4d7a-4ba2-ba29-61eff748e5b8 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:24.370383 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad312d-4d7a-4ba2-ba29-61eff748e5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:24.371024 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad312d-4d7a-4ba2-ba29-61eff748e5b8 None admin] PATH_INFO: `/v3/projects/d7c37d24ebc147969c92537604aabc6b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:24.372354 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad312d-4d7a-4ba2-ba29-61eff748e5b8 None admin] RBAC: Authorizing `identity:get_project(project_id=d7c37d24ebc147969c92537604aabc6b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:24.374007 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad312d-4d7a-4ba2-ba29-61eff748e5b8 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:24.376688 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 95/190] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:24 2022] GET /identity/v3/projects/d7c37d24ebc147969c92537604aabc6b => generated 312 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:24.381830 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad312d-4d7a-4ba2-ba29-61eff748e5b8 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:24.451832 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1fae9-4783-4e12-aa9f-b9c3d614b30f None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:24.488173 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d2f9f-70b0-4af9-8cf2-e12f6997f3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:24.762826 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d2f9f-70b0-4af9-8cf2-e12f6997f3bd None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:24.762826 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d2f9f-70b0-4af9-8cf2-e12f6997f3bd None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:24.762826 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d2f9f-70b0-4af9-8cf2-e12f6997f3bd None admin] PATH_INFO: `/v3/users/0771c65623834672a441c17da5cc4476` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:24.763148 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad312d-4d7a-4ba2-ba29-61eff748e5b8 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:24.763251 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d2f9f-70b0-4af9-8cf2-e12f6997f3bd None admin] RBAC: Authorizing `identity:get_user(user_id=0771c65623834672a441c17da5cc4476)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:24.763251 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d2f9f-70b0-4af9-8cf2-e12f6997f3bd None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:24.763251 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 96/191] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:24 2022] GET /identity/v3/users/0771c65623834672a441c17da5cc4476 => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:24.763251 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d2f9f-70b0-4af9-8cf2-e12f6997f3bd None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:24.763570 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a3d90-a523-401a-b95b-db162a41bd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:24.763570 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a3d90-a523-401a-b95b-db162a41bd57 None admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:24.763570 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a3d90-a523-401a-b95b-db162a41bd57 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:24.763570 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a3d90-a523-401a-b95b-db162a41bd57 None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:24.763902 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a3d90-a523-401a-b95b-db162a41bd57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:24.763902 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a3d90-a523-401a-b95b-db162a41bd57 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:24.764083 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:24.764083 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d2f9f-70b0-4af9-8cf2-e12f6997f3bd None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:24.764083 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627050b-58a0-4694-97e9-728d2bad5c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:24.764083 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627050b-58a0-4694-97e9-728d2bad5c3a None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:24.764408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627050b-58a0-4694-97e9-728d2bad5c3a None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:24.764408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627050b-58a0-4694-97e9-728d2bad5c3a None admin] PATH_INFO: `/v3/users/0771c65623834672a441c17da5cc4476` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:24.764408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627050b-58a0-4694-97e9-728d2bad5c3a None admin] RBAC: Authorizing `identity:get_user(user_id=0771c65623834672a441c17da5cc4476)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:24.764408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627050b-58a0-4694-97e9-728d2bad5c3a None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:24.764408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 97/192] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:24 2022] GET /identity/v3/users/0771c65623834672a441c17da5cc4476 => generated 283 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:24.764408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627050b-58a0-4694-97e9-728d2bad5c3a None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:24.764895 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627050b-58a0-4694-97e9-728d2bad5c3a None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:24.781876 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ae3d8-f7be-44b0-aa39-997f68c52205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:24.782488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ae3d8-f7be-44b0-aa39-997f68c52205 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:24.782769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ae3d8-f7be-44b0-aa39-997f68c52205 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:24.783608 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ae3d8-f7be-44b0-aa39-997f68c52205 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:24.784342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322ae3d8-f7be-44b0-aa39-997f68c52205 None admin] RBAC: Adding query filter params (role.id=5032636d8729461c9926a59b9bcd041b, scope.project.id=d7c37d24ebc147969c92537604aabc6b, user.id=b92a689fcc4842b3aa1d3a5c3f5e47b4) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:24.784680 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322ae3d8-f7be-44b0-aa39-997f68c52205 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:24.786161 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322ae3d8-f7be-44b0-aa39-997f68c52205 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:24.791717 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 98/193] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:34:24 2022] GET /identity/v3/role_assignments?user.id=b92a689fcc4842b3aa1d3a5c3f5e47b4&scope.project.id=d7c37d24ebc147969c92537604aabc6b&role.id=5032636d8729461c9926a59b9bcd041b => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:24.792533 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ae3d8-f7be-44b0-aa39-997f68c52205 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:24.839559 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 96/194] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Aug 26 00:34:24 2022] POST /identity/v3/users => generated 297 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:34:24.840949 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a3d90-a523-401a-b95b-db162a41bd57 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:24.885889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:24.903967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:24.942045 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ae3d8-f7be-44b0-aa39-997f68c52205 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:24.981048 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247697dd-a3f6-47ab-9df8-485b12063969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:24.981703 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247697dd-a3f6-47ab-9df8-485b12063969 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:24.981959 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247697dd-a3f6-47ab-9df8-485b12063969 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:24.982233 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247697dd-a3f6-47ab-9df8-485b12063969 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:25.044238 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a3d90-a523-401a-b95b-db162a41bd57 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:25.044920 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:25.064971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247697dd-a3f6-47ab-9df8-485b12063969 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:25.067159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247697dd-a3f6-47ab-9df8-485b12063969 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:25.093141 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95475e8-f855-4883-b5f3-f422eb38d50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:25.093765 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95475e8-f855-4883-b5f3-f422eb38d50c None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:25.094042 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95475e8-f855-4883-b5f3-f422eb38d50c None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:25.094311 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95475e8-f855-4883-b5f3-f422eb38d50c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:25.186232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95475e8-f855-4883-b5f3-f422eb38d50c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:25.187710 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95475e8-f855-4883-b5f3-f422eb38d50c None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:25.205490 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 99/195] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:34:25.213020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247697dd-a3f6-47ab-9df8-485b12063969 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:25.213820 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:25.318105 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:25.319890 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 97/196] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:34:25.321068 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94182999-455d-4d77-8aa8-353e521fec70 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:25.321463 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94182999-455d-4d77-8aa8-353e521fec70 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:25.321742 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94182999-455d-4d77-8aa8-353e521fec70 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:25.323138 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 98/197] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:25.329157 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51965ca-8ec5-4daf-b977-0beb17eca90f None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:25.331398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51965ca-8ec5-4daf-b977-0beb17eca90f None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:25.331398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51965ca-8ec5-4daf-b977-0beb17eca90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:25.343014 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247697dd-a3f6-47ab-9df8-485b12063969 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:25.350617 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-c51965ca-8ec5-4daf-b977-0beb17eca90f None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:25.352042 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-c51965ca-8ec5-4daf-b977-0beb17eca90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:25.387615 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fce827-9bad-45fa-b31b-43ba606e956f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:25.388098 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fce827-9bad-45fa-b31b-43ba606e956f None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:25.388375 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fce827-9bad-45fa-b31b-43ba606e956f None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:25.388697 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fce827-9bad-45fa-b31b-43ba606e956f None admin] PATH_INFO: `/v3/projects/22414db73cef49549c4930bec9f6b5dc` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:25.389966 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fce827-9bad-45fa-b31b-43ba606e956f None admin] RBAC: Authorizing `identity:get_project(project_id=22414db73cef49549c4930bec9f6b5dc)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:25.393257 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fce827-9bad-45fa-b31b-43ba606e956f None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:25.395212 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 100/198] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:25 2022] GET /identity/v3/projects/22414db73cef49549c4930bec9f6b5dc => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:25.396003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fce827-9bad-45fa-b31b-43ba606e956f None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:25.456546 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 99/199] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:34:25.458480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51965ca-8ec5-4daf-b977-0beb17eca90f None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:25.534089 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:25.562664 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fce827-9bad-45fa-b31b-43ba606e956f None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:25.579518 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51965ca-8ec5-4daf-b977-0beb17eca90f None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:25.624327 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb2680-f30e-42cf-a99d-1df3f901d1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:25.624968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb2680-f30e-42cf-a99d-1df3f901d1ac None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:25.625245 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb2680-f30e-42cf-a99d-1df3f901d1ac None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:25.625511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb2680-f30e-42cf-a99d-1df3f901d1ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:25.626246 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb2680-f30e-42cf-a99d-1df3f901d1ac None admin] RBAC: Adding query filter params (role.id=1a33e3c8cb8347feb9498595886cc621, scope.project.id=22414db73cef49549c4930bec9f6b5dc, user.id=0771c65623834672a441c17da5cc4476) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:25.626547 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb2680-f30e-42cf-a99d-1df3f901d1ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:25.628040 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb2680-f30e-42cf-a99d-1df3f901d1ac None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:25.636754 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701f79e-9315-4c50-a7c6-2d25f28a7826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:25.637395 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701f79e-9315-4c50-a7c6-2d25f28a7826 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:25.637673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701f79e-9315-4c50-a7c6-2d25f28a7826 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:25.637946 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701f79e-9315-4c50-a7c6-2d25f28a7826 None admin] PATH_INFO: `/v3/projects/22414db73cef49549c4930bec9f6b5dc` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:25.639398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4701f79e-9315-4c50-a7c6-2d25f28a7826 None admin] RBAC: Authorizing `identity:get_project(project_id=22414db73cef49549c4930bec9f6b5dc)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:25.640866 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4701f79e-9315-4c50-a7c6-2d25f28a7826 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:25.643499 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 101/200] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:25 2022] GET /identity/v3/projects/22414db73cef49549c4930bec9f6b5dc => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:25.649146 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701f79e-9315-4c50-a7c6-2d25f28a7826 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:25.650062 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 100/201] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:34:25 2022] GET /identity/v3/role_assignments?user.id=0771c65623834672a441c17da5cc4476&scope.project.id=22414db73cef49549c4930bec9f6b5dc&role.id=1a33e3c8cb8347feb9498595886cc621 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:25.682078 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:25.788680 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701f79e-9315-4c50-a7c6-2d25f28a7826 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:25.794040 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb2680-f30e-42cf-a99d-1df3f901d1ac None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:25.849276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d264b3-bb1b-41a1-a525-dbc99942be0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:25.849815 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d264b3-bb1b-41a1-a525-dbc99942be0e None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:25.850099 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d264b3-bb1b-41a1-a525-dbc99942be0e None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:25.850369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d264b3-bb1b-41a1-a525-dbc99942be0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:25.851100 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d264b3-bb1b-41a1-a525-dbc99942be0e None admin] RBAC: Adding query filter params (role.id=423aa6ac8b05402da7587e83e5c4ce09, scope.project.id=22414db73cef49549c4930bec9f6b5dc, user.id=0771c65623834672a441c17da5cc4476) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:25.851407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d264b3-bb1b-41a1-a525-dbc99942be0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:25.852907 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d264b3-bb1b-41a1-a525-dbc99942be0e None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:25.858922 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 102/202] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:34:25 2022] GET /identity/v3/role_assignments?user.id=0771c65623834672a441c17da5cc4476&scope.project.id=22414db73cef49549c4930bec9f6b5dc&role.id=423aa6ac8b05402da7587e83e5c4ce09 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:25.930804 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb2680-f30e-42cf-a99d-1df3f901d1ac None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:25.949576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:25.968569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825864b0-65ef-41d9-baab-785a1371d2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:25.969608 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825864b0-65ef-41d9-baab-785a1371d2a8 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:25.969894 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825864b0-65ef-41d9-baab-785a1371d2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:25.970166 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825864b0-65ef-41d9-baab-785a1371d2a8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:25.982508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825864b0-65ef-41d9-baab-785a1371d2a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:25.987990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-825864b0-65ef-41d9-baab-785a1371d2a8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 00:34:25.987990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:25.987990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:25.987990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:25.987990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:25.987990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:25.987990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:25.987990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:25.987990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:25.987990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:25.987990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:25.987990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:25.987990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:25.987990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:25.987990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:25.987990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:25.989018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:25.989018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:25.989018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:25.989018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:25.989018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:25.989018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:25.989018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:25.989018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:25.989018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:25.989018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:25.989018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:25.989018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:25.989018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:25.989018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:25.989018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:25.989018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:25.989868 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:25.989868 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:25.989868 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:25.989868 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:25.989868 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:25.989868 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:25.989868 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:25.989868 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:25.989868 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:25.989868 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:25.989868 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 00:34:25.989868 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:34:25.989868 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 101/203] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:34:25.997690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d264b3-bb1b-41a1-a525-dbc99942be0e None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:26.157468 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:26.176211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d264b3-bb1b-41a1-a525-dbc99942be0e None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:26.223075 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fefb80-3e90-4d0c-9a07-c8c7b05c92a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:26.223728 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fefb80-3e90-4d0c-9a07-c8c7b05c92a4 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:26.223963 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fefb80-3e90-4d0c-9a07-c8c7b05c92a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:26.224241 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fefb80-3e90-4d0c-9a07-c8c7b05c92a4 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:26.224945 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fefb80-3e90-4d0c-9a07-c8c7b05c92a4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:26.225248 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fefb80-3e90-4d0c-9a07-c8c7b05c92a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:26.226626 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fefb80-3e90-4d0c-9a07-c8c7b05c92a4 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:26.235831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 103/204] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:34:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:26.241201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825864b0-65ef-41d9-baab-785a1371d2a8 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:26.404277 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825864b0-65ef-41d9-baab-785a1371d2a8 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:26.466099 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f10005-36bc-43c0-9ab9-ffc252077ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:26.466755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f10005-36bc-43c0-9ab9-ffc252077ef9 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:26.467035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f10005-36bc-43c0-9ab9-ffc252077ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:26.467310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f10005-36bc-43c0-9ab9-ffc252077ef9 None admin] PATH_INFO: `/v3/users/f52fa6565d874f4e9d40b577d033f854` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:26.468455 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f10005-36bc-43c0-9ab9-ffc252077ef9 None admin] RBAC: Authorizing `identity:get_user(user_id=f52fa6565d874f4e9d40b577d033f854)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:26.469925 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f10005-36bc-43c0-9ab9-ffc252077ef9 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:26.471542 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 102/205] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:26 2022] GET /identity/v3/users/f52fa6565d874f4e9d40b577d033f854 => generated 289 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:26.477412 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fefb80-3e90-4d0c-9a07-c8c7b05c92a4 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:26.584290 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fefb80-3e90-4d0c-9a07-c8c7b05c92a4 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:26.631642 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced323c-4d9c-44ff-94ab-6c30f3bc0956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:26.632275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced323c-4d9c-44ff-94ab-6c30f3bc0956 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:26.632630 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced323c-4d9c-44ff-94ab-6c30f3bc0956 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:26.632910 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced323c-4d9c-44ff-94ab-6c30f3bc0956 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:26.653764 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:26.720593 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ced323c-4d9c-44ff-94ab-6c30f3bc0956 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:26.721994 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ced323c-4d9c-44ff-94ab-6c30f3bc0956 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:26.825608 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 104/206] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:34:26.831337 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f10005-36bc-43c0-9ab9-ffc252077ef9 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:26.896215 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:26.985201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f10005-36bc-43c0-9ab9-ffc252077ef9 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:27.040488 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc3eb0-2216-4183-9b83-ba048a3846e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:27.041145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc3eb0-2216-4183-9b83-ba048a3846e3 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:27.041426 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc3eb0-2216-4183-9b83-ba048a3846e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:27.041699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc3eb0-2216-4183-9b83-ba048a3846e3 None admin] PATH_INFO: `/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:27.042844 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc3eb0-2216-4183-9b83-ba048a3846e3 None admin] RBAC: Authorizing `identity:get_project(project_id=c60b6aa5591d4178ba9eb5fd05b7ee73)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:27.044303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc3eb0-2216-4183-9b83-ba048a3846e3 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:27.046367 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 103/207] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:26 2022] GET /identity/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:27.052810 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced323c-4d9c-44ff-94ab-6c30f3bc0956 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:27.131945 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:27.192308 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced323c-4d9c-44ff-94ab-6c30f3bc0956 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:27.238928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b89648-32e0-473f-9a6b-aff5eebbe4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:27.239542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b89648-32e0-473f-9a6b-aff5eebbe4c1 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:27.239771 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b89648-32e0-473f-9a6b-aff5eebbe4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:27.239999 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b89648-32e0-473f-9a6b-aff5eebbe4c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:27.240763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b89648-32e0-473f-9a6b-aff5eebbe4c1 None admin] RBAC: Adding query filter params (role.id=b68669d0f20b4ac8a1a7d46142b03424, scope.project.id=c60b6aa5591d4178ba9eb5fd05b7ee73, user.id=f52fa6565d874f4e9d40b577d033f854) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:27.241074 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b89648-32e0-473f-9a6b-aff5eebbe4c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:27.242515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b89648-32e0-473f-9a6b-aff5eebbe4c1 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:27.248788 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 105/208] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:34:27 2022] GET /identity/v3/role_assignments?user.id=f52fa6565d874f4e9d40b577d033f854&scope.project.id=c60b6aa5591d4178ba9eb5fd05b7ee73&role.id=b68669d0f20b4ac8a1a7d46142b03424 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:27.427781 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ab65e-f8f1-4320-a199-3aaec8f416b5 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:27.428314 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ab65e-f8f1-4320-a199-3aaec8f416b5 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:27.428314 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ab65e-f8f1-4320-a199-3aaec8f416b5 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:27.429766 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 104/209] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:27.435443 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b3768-7186-44d9-8882-5dc9933a47c9 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:27.436032 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b3768-7186-44d9-8882-5dc9933a47c9 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:27.436795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b3768-7186-44d9-8882-5dc9933a47c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:27.464765 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-405b3768-7186-44d9-8882-5dc9933a47c9 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:27.466006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-405b3768-7186-44d9-8882-5dc9933a47c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:27.548585 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 106/210] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:27.806296 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0514335-ef1f-4254-b0f3-07ce1712d6f9 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:27.806485 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0514335-ef1f-4254-b0f3-07ce1712d6f9 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:27.806755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0514335-ef1f-4254-b0f3-07ce1712d6f9 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:27.808379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 105/211] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:27.813807 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fb3b3-cf8c-473c-b511-00254cb0b215 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:27.814805 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fb3b3-cf8c-473c-b511-00254cb0b215 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:27.815182 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fb3b3-cf8c-473c-b511-00254cb0b215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:27.832881 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0514335-ef1f-4254-b0f3-07ce1712d6f9 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:27.835431 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-ab3fb3b3-cf8c-473c-b511-00254cb0b215 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:27.836879 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-ab3fb3b3-cf8c-473c-b511-00254cb0b215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:27.880070 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:27.927887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 107/212] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:27.961690 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0514335-ef1f-4254-b0f3-07ce1712d6f9 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:27.977107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:27.999085 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf953e-04a6-42b3-a4d2-065527ab068e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:27.999977 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf953e-04a6-42b3-a4d2-065527ab068e None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:28.000407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf953e-04a6-42b3-a4d2-065527ab068e None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:28.000850 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf953e-04a6-42b3-a4d2-065527ab068e None admin] PATH_INFO: `/v3/roles/member` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:28.012446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf953e-04a6-42b3-a4d2-065527ab068e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:28.017807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-44bf953e-04a6-42b3-a4d2-065527ab068e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:28.017807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:28.017807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:28.017807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:28.017807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:28.017807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:28.017807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:28.017807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:28.017807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:28.017807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:28.017807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:28.017807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:28.017807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:28.017807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:28.017807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:28.017807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:28.019507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:28.019507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:28.019507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:28.019507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:28.019507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:28.019507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:28.019507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:28.019507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:28.019507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:28.019507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:28.019507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:28.019507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:28.019507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:28.019507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:28.019507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:28.019507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:28.020797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:28.020797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:28.020797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:28.020797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:28.020797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:28.020797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:28.020797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:28.020797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:28.020797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:28.020797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:28.020797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:28.020797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:34:28.023208 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 106/213] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:34:28.031465 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3fb3b3-cf8c-473c-b511-00254cb0b215 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:28.062982 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f49f8-f1a4-4999-a41b-6c5917b603df None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:28.063269 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f49f8-f1a4-4999-a41b-6c5917b603df None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:28.063546 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f49f8-f1a4-4999-a41b-6c5917b603df None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:28.065072 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 107/214] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:28.069772 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002aefb-60c4-45d9-adc2-865659966882 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:28.070058 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002aefb-60c4-45d9-adc2-865659966882 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:28.070872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002aefb-60c4-45d9-adc2-865659966882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:28.094317 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-1002aefb-60c4-45d9-adc2-865659966882 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:28.096190 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-1002aefb-60c4-45d9-adc2-865659966882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:28.172294 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 108/215] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:34:28.174113 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241d7c1-3563-410d-a892-680c8eebe10b None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:28.174404 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241d7c1-3563-410d-a892-680c8eebe10b None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:28.174679 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241d7c1-3563-410d-a892-680c8eebe10b None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:28.176080 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 109/216] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:28.181273 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697c028-b84d-4ca6-848f-73396bc096fa None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:28.181562 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697c028-b84d-4ca6-848f-73396bc096fa None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:28.181843 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697c028-b84d-4ca6-848f-73396bc096fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:28.200630 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-6697c028-b84d-4ca6-848f-73396bc096fa None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:28.201879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-6697c028-b84d-4ca6-848f-73396bc096fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:28.202284 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3fb3b3-cf8c-473c-b511-00254cb0b215 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:28.242836 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a58dee-13b1-4a1f-97c0-c0363c4e8290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:28.243452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a58dee-13b1-4a1f-97c0-c0363c4e8290 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:28.243809 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a58dee-13b1-4a1f-97c0-c0363c4e8290 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:28.243982 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a58dee-13b1-4a1f-97c0-c0363c4e8290 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:28.245126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a58dee-13b1-4a1f-97c0-c0363c4e8290 None admin] RBAC: Adding query filter params (name=member) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:28.245481 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a58dee-13b1-4a1f-97c0-c0363c4e8290 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:28.247251 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a58dee-13b1-4a1f-97c0-c0363c4e8290 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:28.256019 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 108/217] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:34:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:28.260511 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a58dee-13b1-4a1f-97c0-c0363c4e8290 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:28.273945 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 110/218] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:34:28.274767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697c028-b84d-4ca6-848f-73396bc096fa None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:28.300386 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:28.321950 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:28.370382 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a58dee-13b1-4a1f-97c0-c0363c4e8290 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:28.410418 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697c028-b84d-4ca6-848f-73396bc096fa None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:28.413711 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483437e1-8b1a-4741-bf32-319c69ec7d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:28.414353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483437e1-8b1a-4741-bf32-319c69ec7d02 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:28.414632 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483437e1-8b1a-4741-bf32-319c69ec7d02 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:28.414908 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483437e1-8b1a-4741-bf32-319c69ec7d02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:28.416019 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483437e1-8b1a-4741-bf32-319c69ec7d02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:28.417508 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483437e1-8b1a-4741-bf32-319c69ec7d02 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:28.419379 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 109/219] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:34:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:28.425589 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483437e1-8b1a-4741-bf32-319c69ec7d02 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:28.453124 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47cdea8-756f-4a93-b0ea-51f87bcb7469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:28.453848 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47cdea8-756f-4a93-b0ea-51f87bcb7469 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:28.454198 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47cdea8-756f-4a93-b0ea-51f87bcb7469 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:28.454477 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47cdea8-756f-4a93-b0ea-51f87bcb7469 None admin] PATH_INFO: `/v3/users/a607948d7f414d0caf2eaee9eb0ec330` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:28.469783 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47cdea8-756f-4a93-b0ea-51f87bcb7469 None admin] RBAC: Authorizing `identity:get_user(user_id=a607948d7f414d0caf2eaee9eb0ec330)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:28.472028 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47cdea8-756f-4a93-b0ea-51f87bcb7469 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:28.476807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 111/220] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:28 2022] GET /identity/v3/users/a607948d7f414d0caf2eaee9eb0ec330 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:28.476807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47cdea8-756f-4a93-b0ea-51f87bcb7469 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:28.535061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:28.538611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483437e1-8b1a-4741-bf32-319c69ec7d02 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:28.565592 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:28.574258 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47cdea8-756f-4a93-b0ea-51f87bcb7469 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:28.575250 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150df08b-e6f0-4f28-b967-7fce9256f11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:28.576167 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150df08b-e6f0-4f28-b967-7fce9256f11f None admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:28.576454 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150df08b-e6f0-4f28-b967-7fce9256f11f None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:28.576754 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150df08b-e6f0-4f28-b967-7fce9256f11f None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:28.577411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150df08b-e6f0-4f28-b967-7fce9256f11f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:28.578884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150df08b-e6f0-4f28-b967-7fce9256f11f None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:28.608342 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003be0f-58f4-422d-bb57-448bc853ac54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:28.609126 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003be0f-58f4-422d-bb57-448bc853ac54 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:28.609408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003be0f-58f4-422d-bb57-448bc853ac54 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:28.609679 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003be0f-58f4-422d-bb57-448bc853ac54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:28.619582 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a003be0f-58f4-422d-bb57-448bc853ac54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:28.624319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-a003be0f-58f4-422d-bb57-448bc853ac54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:34:28.624319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:28.624319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:28.624319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:28.624319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:28.624319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:28.624319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:28.624319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:28.624319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:28.624319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:28.624319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:28.624319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:28.624319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:28.624319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:28.624319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:28.624319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:28.625437 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:28.625437 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:28.625437 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:28.625437 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:28.625437 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:28.625437 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:28.625437 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:28.625437 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:28.625437 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:28.625437 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:28.625437 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:28.625437 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:28.625437 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:28.625437 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:28.625437 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:28.625437 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:28.626416 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:28.626416 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:28.626416 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:28.626416 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:28.626416 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:28.626416 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:28.626416 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:28.626416 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:28.626416 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:28.626416 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:28.626416 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:34:28.626416 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:34:28.626416 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 112/221] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:34:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:28.626416 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003be0f-58f4-422d-bb57-448bc853ac54 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:28.666466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 110/222] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Aug 26 00:34:28 2022] POST /identity/v3/users => generated 297 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:34:28.667614 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150df08b-e6f0-4f28-b967-7fce9256f11f None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:28.736124 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003be0f-58f4-422d-bb57-448bc853ac54 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:28.775675 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3fe9f-6301-4d79-afdc-2e5420a51a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:28.776326 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3fe9f-6301-4d79-afdc-2e5420a51a2f None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:28.776810 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3fe9f-6301-4d79-afdc-2e5420a51a2f None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:28.777076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3fe9f-6301-4d79-afdc-2e5420a51a2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:28.789292 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150df08b-e6f0-4f28-b967-7fce9256f11f None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:28.802407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:28.833819 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e075d18-3df6-4b93-8fd2-6dae2db9348d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:28.834391 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e075d18-3df6-4b93-8fd2-6dae2db9348d None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:28.834669 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e075d18-3df6-4b93-8fd2-6dae2db9348d None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:28.834945 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e075d18-3df6-4b93-8fd2-6dae2db9348d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:28.846921 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e075d18-3df6-4b93-8fd2-6dae2db9348d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:28.852173 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-0e075d18-3df6-4b93-8fd2-6dae2db9348d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 00:34:28.852173 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:28.852173 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:28.852173 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:28.852173 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:28.852173 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:28.852173 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:28.852173 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:28.852173 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:28.852173 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:28.852173 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:28.852173 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:28.852173 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:28.852173 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:28.852173 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:28.852173 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:28.853211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:28.853211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:28.853211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:28.853211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:28.853211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:28.853211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:28.853211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:28.853211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:28.853211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:28.853211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:28.853211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:28.853211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:28.853211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:28.853211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:28.853211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:28.853211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:28.854020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:28.854020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:28.854020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:28.854020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:28.854020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:28.854020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:28.854020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:28.854020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:28.854020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:28.854020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:28.854020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 00:34:28.854020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:34:28.854020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 111/223] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Aug 26 00:34:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:28.854897 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e075d18-3df6-4b93-8fd2-6dae2db9348d None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:28.867416 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e3fe9f-6301-4d79-afdc-2e5420a51a2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:28.869021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e3fe9f-6301-4d79-afdc-2e5420a51a2f None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:28.876216 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:28.987686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e075d18-3df6-4b93-8fd2-6dae2db9348d None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:28.996868 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 113/224] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:34:28.997739 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3fe9f-6301-4d79-afdc-2e5420a51a2f None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:29.034061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494fe240-2f11-416b-b5ea-585198644531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:29.034987 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494fe240-2f11-416b-b5ea-585198644531 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:29.035474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494fe240-2f11-416b-b5ea-585198644531 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:29.036069 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494fe240-2f11-416b-b5ea-585198644531 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:29.037797 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494fe240-2f11-416b-b5ea-585198644531 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:29.038321 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494fe240-2f11-416b-b5ea-585198644531 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:29.039967 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494fe240-2f11-416b-b5ea-585198644531 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:29.058004 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 112/225] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:29.058004 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494fe240-2f11-416b-b5ea-585198644531 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:29.062453 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:29.115445 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3fe9f-6301-4d79-afdc-2e5420a51a2f None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:29.151961 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b33cb-ad00-423f-bccf-e8410e2b3e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:29.152780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b33cb-ad00-423f-bccf-e8410e2b3e0a None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:29.153142 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b33cb-ad00-423f-bccf-e8410e2b3e0a None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:29.153798 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b33cb-ad00-423f-bccf-e8410e2b3e0a None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:29.155263 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b33cb-ad00-423f-bccf-e8410e2b3e0a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:29.155263 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b33cb-ad00-423f-bccf-e8410e2b3e0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:29.156490 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b33cb-ad00-423f-bccf-e8410e2b3e0a None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:29.169948 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 114/226] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:34:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:29.170835 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:29.171182 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b33cb-ad00-423f-bccf-e8410e2b3e0a None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:29.193996 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494fe240-2f11-416b-b5ea-585198644531 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:29.240793 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0103582-a0c8-4d13-b92b-ad86240518d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:29.242152 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0103582-a0c8-4d13-b92b-ad86240518d2 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:29.242433 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0103582-a0c8-4d13-b92b-ad86240518d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:29.242708 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0103582-a0c8-4d13-b92b-ad86240518d2 None admin] PATH_INFO: `/v3/projects/22414db73cef49549c4930bec9f6b5dc` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:29.244391 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0103582-a0c8-4d13-b92b-ad86240518d2 None admin] RBAC: Authorizing `identity:get_project(project_id=22414db73cef49549c4930bec9f6b5dc)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:29.246096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0103582-a0c8-4d13-b92b-ad86240518d2 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:29.253353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 113/227] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:29 2022] GET /identity/v3/projects/22414db73cef49549c4930bec9f6b5dc => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:29.253353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0103582-a0c8-4d13-b92b-ad86240518d2 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:29.297094 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b33cb-ad00-423f-bccf-e8410e2b3e0a None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:29.313584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:29.335583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55751b8-b910-4dd2-b82f-5c0c64da063a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:29.336368 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55751b8-b910-4dd2-b82f-5c0c64da063a None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:29.336818 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55751b8-b910-4dd2-b82f-5c0c64da063a None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:29.337169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55751b8-b910-4dd2-b82f-5c0c64da063a None admin] PATH_INFO: `/v3/users/0771c65623834672a441c17da5cc4476` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:29.338485 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55751b8-b910-4dd2-b82f-5c0c64da063a None admin] RBAC: Authorizing `identity:get_user(user_id=0771c65623834672a441c17da5cc4476)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:29.340146 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55751b8-b910-4dd2-b82f-5c0c64da063a None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:29.342274 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 115/228] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:29 2022] GET /identity/v3/users/0771c65623834672a441c17da5cc4476 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:29.343233 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55751b8-b910-4dd2-b82f-5c0c64da063a None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:29.378012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0103582-a0c8-4d13-b92b-ad86240518d2 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:29.422073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d96eb0-12fa-4dbb-a568-3aef1fa8532c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:29.422880 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d96eb0-12fa-4dbb-a568-3aef1fa8532c None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:29.423257 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d96eb0-12fa-4dbb-a568-3aef1fa8532c None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:29.424153 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d96eb0-12fa-4dbb-a568-3aef1fa8532c None admin] PATH_INFO: `/v3/users/0771c65623834672a441c17da5cc4476` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:29.425446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d96eb0-12fa-4dbb-a568-3aef1fa8532c None admin] RBAC: Authorizing `identity:get_user(user_id=0771c65623834672a441c17da5cc4476)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:29.427119 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d96eb0-12fa-4dbb-a568-3aef1fa8532c None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:29.428905 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 114/229] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:29 2022] GET /identity/v3/users/0771c65623834672a441c17da5cc4476 => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:29.431398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d96eb0-12fa-4dbb-a568-3aef1fa8532c None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:29.456389 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:29.482893 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55751b8-b910-4dd2-b82f-5c0c64da063a None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:29.526458 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9198df-c044-45b9-ac02-080192f5c65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:29.527365 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9198df-c044-45b9-ac02-080192f5c65d None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:29.527798 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9198df-c044-45b9-ac02-080192f5c65d None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:29.528195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9198df-c044-45b9-ac02-080192f5c65d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:29.529214 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9198df-c044-45b9-ac02-080192f5c65d None admin] RBAC: Adding query filter params (role.id=5032636d8729461c9926a59b9bcd041b, scope.project.id=22414db73cef49549c4930bec9f6b5dc, user.id=a607948d7f414d0caf2eaee9eb0ec330) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:29.529524 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9198df-c044-45b9-ac02-080192f5c65d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:29.531488 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9198df-c044-45b9-ac02-080192f5c65d None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:29.539234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 116/230] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:34:29 2022] GET /identity/v3/role_assignments?user.id=a607948d7f414d0caf2eaee9eb0ec330&scope.project.id=22414db73cef49549c4930bec9f6b5dc&role.id=5032636d8729461c9926a59b9bcd041b => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:29.539935 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9198df-c044-45b9-ac02-080192f5c65d None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:29.575394 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d96eb0-12fa-4dbb-a568-3aef1fa8532c None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:29.596605 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:29.659624 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9198df-c044-45b9-ac02-080192f5c65d None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:29.661285 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176938c0-42c3-44d4-bea9-bc8a0d9dc120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:29.662011 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176938c0-42c3-44d4-bea9-bc8a0d9dc120 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:29.662294 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176938c0-42c3-44d4-bea9-bc8a0d9dc120 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:29.662571 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176938c0-42c3-44d4-bea9-bc8a0d9dc120 None admin] PATH_INFO: `/v3/projects/22414db73cef49549c4930bec9f6b5dc` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:29.663780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176938c0-42c3-44d4-bea9-bc8a0d9dc120 None admin] RBAC: Authorizing `identity:get_project(project_id=22414db73cef49549c4930bec9f6b5dc)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:29.665273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176938c0-42c3-44d4-bea9-bc8a0d9dc120 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:29.667062 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 115/231] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:29 2022] GET /identity/v3/projects/22414db73cef49549c4930bec9f6b5dc => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:29.673340 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176938c0-42c3-44d4-bea9-bc8a0d9dc120 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:29.704295 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ab9f4-6a9b-4149-a6e6-82bb55414480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:29.706271 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ab9f4-6a9b-4149-a6e6-82bb55414480 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:29.708381 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ab9f4-6a9b-4149-a6e6-82bb55414480 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:29.708870 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ab9f4-6a9b-4149-a6e6-82bb55414480 None admin] PATH_INFO: `/v3/projects/22414db73cef49549c4930bec9f6b5dc` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:29.711952 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ab9f4-6a9b-4149-a6e6-82bb55414480 None admin] RBAC: Authorizing `identity:get_project(project_id=22414db73cef49549c4930bec9f6b5dc)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:29.714784 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ab9f4-6a9b-4149-a6e6-82bb55414480 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:29.721170 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 117/232] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:29 2022] GET /identity/v3/projects/22414db73cef49549c4930bec9f6b5dc => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:29.724369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ab9f4-6a9b-4149-a6e6-82bb55414480 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:29.827699 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:29.849102 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176938c0-42c3-44d4-bea9-bc8a0d9dc120 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:29.889327 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905aaa1d-6305-440a-a51a-6ead03c7703c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:29.890363 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905aaa1d-6305-440a-a51a-6ead03c7703c None admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:29.890862 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905aaa1d-6305-440a-a51a-6ead03c7703c None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:29.891281 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905aaa1d-6305-440a-a51a-6ead03c7703c None admin] PATH_INFO: `/v3/projects/22414db73cef49549c4930bec9f6b5dc/users/0771c65623834672a441c17da5cc4476/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:29.894018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905aaa1d-6305-440a-a51a-6ead03c7703c None admin] RBAC: Authorizing `identity:create_grant(project_id=22414db73cef49549c4930bec9f6b5dc, user_id=0771c65623834672a441c17da5cc4476, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:29.895626 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905aaa1d-6305-440a-a51a-6ead03c7703c None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:29.931432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ab9f4-6a9b-4149-a6e6-82bb55414480 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:29.948087 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:29.972414 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308209bc-4ef7-426f-a492-bfd47a984ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:29.973379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308209bc-4ef7-426f-a492-bfd47a984ba2 None admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:29.973805 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308209bc-4ef7-426f-a492-bfd47a984ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:29.974259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308209bc-4ef7-426f-a492-bfd47a984ba2 None admin] PATH_INFO: `/v3/projects/22414db73cef49549c4930bec9f6b5dc/users/0771c65623834672a441c17da5cc4476/roles/423aa6ac8b05402da7587e83e5c4ce09` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:29.976824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308209bc-4ef7-426f-a492-bfd47a984ba2 None admin] RBAC: Authorizing `identity:create_grant(project_id=22414db73cef49549c4930bec9f6b5dc, user_id=0771c65623834672a441c17da5cc4476, role_id=423aa6ac8b05402da7587e83e5c4ce09)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:29.978427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308209bc-4ef7-426f-a492-bfd47a984ba2 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:29.987129 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 116/233] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Aug 26 00:34:29 2022] PUT /identity/v3/projects/22414db73cef49549c4930bec9f6b5dc/users/0771c65623834672a441c17da5cc4476/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:34:30.056428 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 118/234] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Aug 26 00:34:29 2022] PUT /identity/v3/projects/22414db73cef49549c4930bec9f6b5dc/users/0771c65623834672a441c17da5cc4476/roles/423aa6ac8b05402da7587e83e5c4ce09 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:34:30.873681 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba50b77-8b6d-4e00-a5f7-e76d62d28442 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:30.874886 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba50b77-8b6d-4e00-a5f7-e76d62d28442 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:30.875452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba50b77-8b6d-4e00-a5f7-e76d62d28442 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:30.878150 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 117/235] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:30.884163 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe0073-36d6-4c8a-9748-ede8fbaf0351 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:30.885506 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe0073-36d6-4c8a-9748-ede8fbaf0351 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:30.886159 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe0073-36d6-4c8a-9748-ede8fbaf0351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:30.909633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-3dbe0073-36d6-4c8a-9748-ede8fbaf0351 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:30.911189 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-3dbe0073-36d6-4c8a-9748-ede8fbaf0351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:30.958438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ffe38-e1db-458e-99e7-99f5c9751b10 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:30.959042 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ffe38-e1db-458e-99e7-99f5c9751b10 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:30.959532 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ffe38-e1db-458e-99e7-99f5c9751b10 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:30.961244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 118/236] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:34:30.966323 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fed600-19d6-4626-9483-679e4566457e None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:30.966607 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fed600-19d6-4626-9483-679e4566457e None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:30.966881 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fed600-19d6-4626-9483-679e4566457e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:30.997345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-28fed600-19d6-4626-9483-679e4566457e None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:31.000128 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-28fed600-19d6-4626-9483-679e4566457e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:31.016712 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 119/237] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:31.109071 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 119/238] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:31.421074 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe0073-36d6-4c8a-9748-ede8fbaf0351 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:31.457213 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fed600-19d6-4626-9483-679e4566457e None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:31.551936 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe0073-36d6-4c8a-9748-ede8fbaf0351 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:31.556382 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:31.589790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b61c0-200a-446a-8303-6a7caad60787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:31.590786 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b61c0-200a-446a-8303-6a7caad60787 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:31.591257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b61c0-200a-446a-8303-6a7caad60787 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:31.591717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b61c0-200a-446a-8303-6a7caad60787 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:31.612880 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b61c0-200a-446a-8303-6a7caad60787 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:31.618240 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-6f6b61c0-200a-446a-8303-6a7caad60787 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 00:34:31.618240 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:31.618240 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:31.618240 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:31.618240 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:31.618240 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:31.618240 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:31.618240 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:31.618240 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:31.618240 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:31.618240 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:31.618240 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:31.618240 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:31.618240 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:31.618240 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:31.618240 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:31.619169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:31.619169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:31.619169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:31.619169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:31.619169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:31.619169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:31.619169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:31.619169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:31.619169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:31.619169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:31.619169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:31.619169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:31.619169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:31.619169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:31.619169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:31.619169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:31.619954 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:31.619954 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:31.619954 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:31.619954 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:31.619954 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:31.619954 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:31.619954 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:31.619954 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:31.619954 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:31.619954 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:31.619954 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 00:34:31.619954 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:34:31.622521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:31.644230 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 120/239] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:31.654325 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b61c0-200a-446a-8303-6a7caad60787 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:31.664842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fed600-19d6-4626-9483-679e4566457e None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:31.701079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0aea86-5f3a-4078-a945-203958508896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:31.702087 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0aea86-5f3a-4078-a945-203958508896 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:31.702559 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0aea86-5f3a-4078-a945-203958508896 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:31.703039 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0aea86-5f3a-4078-a945-203958508896 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:31.704355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0aea86-5f3a-4078-a945-203958508896 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:31.706162 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0aea86-5f3a-4078-a945-203958508896 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:31.708333 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 120/240] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:34:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:31.717501 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0aea86-5f3a-4078-a945-203958508896 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:31.838682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b61c0-200a-446a-8303-6a7caad60787 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:31.873675 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b653541-3ad4-44c2-a3e2-0f2c5d8a50a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:31.874288 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b653541-3ad4-44c2-a3e2-0f2c5d8a50a1 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:31.874571 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b653541-3ad4-44c2-a3e2-0f2c5d8a50a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:31.874843 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b653541-3ad4-44c2-a3e2-0f2c5d8a50a1 None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:31.875507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b653541-3ad4-44c2-a3e2-0f2c5d8a50a1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:31.875810 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b653541-3ad4-44c2-a3e2-0f2c5d8a50a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:31.877780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b653541-3ad4-44c2-a3e2-0f2c5d8a50a1 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:31.886439 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 121/241] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:34:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:31.892148 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b653541-3ad4-44c2-a3e2-0f2c5d8a50a1 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:31.922875 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0aea86-5f3a-4078-a945-203958508896 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:31.952922 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:31.963298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8aed3-d8c3-4c01-a763-6607e47360b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:31.964159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8aed3-d8c3-4c01-a763-6607e47360b1 None admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:31.964629 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8aed3-d8c3-4c01-a763-6607e47360b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:31.976719 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8aed3-d8c3-4c01-a763-6607e47360b1 None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:31.976719 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8aed3-d8c3-4c01-a763-6607e47360b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:31.976719 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8aed3-d8c3-4c01-a763-6607e47360b1 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:31.979009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:32.006639 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b653541-3ad4-44c2-a3e2-0f2c5d8a50a1 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:32.041378 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef249a-384a-42c2-87d8-2e7e645de65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:32.042004 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef249a-384a-42c2-87d8-2e7e645de65b None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:32.042285 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef249a-384a-42c2-87d8-2e7e645de65b None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:32.042559 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef249a-384a-42c2-87d8-2e7e645de65b None admin] PATH_INFO: `/v3/users/548e7a814507494bb420b223caffaaab` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:32.046576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 121/242] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Aug 26 00:34:31 2022] POST /identity/v3/users => generated 293 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:34:32.050314 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83223263-52da-4711-bda0-f131c43b0697 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:32.051206 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83223263-52da-4711-bda0-f131c43b0697 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:32.051528 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83223263-52da-4711-bda0-f131c43b0697 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:32.057764 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 122/243] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:32.060176 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c864855-939c-421c-9853-15e5d3434c52 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:32.060746 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c864855-939c-421c-9853-15e5d3434c52 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:32.061237 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c864855-939c-421c-9853-15e5d3434c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:32.066258 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef249a-384a-42c2-87d8-2e7e645de65b None admin] RBAC: Authorizing `identity:get_user(user_id=548e7a814507494bb420b223caffaaab)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:32.067918 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef249a-384a-42c2-87d8-2e7e645de65b None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:32.077932 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 122/244] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:31 2022] GET /identity/v3/users/548e7a814507494bb420b223caffaaab => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:32.092824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-4c864855-939c-421c-9853-15e5d3434c52 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:32.093045 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef249a-384a-42c2-87d8-2e7e645de65b None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:32.094552 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-4c864855-939c-421c-9853-15e5d3434c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:32.181530 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 123/245] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:34:32.186052 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:32.202486 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef249a-384a-42c2-87d8-2e7e645de65b None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:32.240322 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980b088-d2fa-46c0-9a50-4e74c53e1f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:32.240898 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980b088-d2fa-46c0-9a50-4e74c53e1f30 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:32.241179 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980b088-d2fa-46c0-9a50-4e74c53e1f30 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:32.241454 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980b088-d2fa-46c0-9a50-4e74c53e1f30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:32.314523 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980b088-d2fa-46c0-9a50-4e74c53e1f30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:32.315994 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980b088-d2fa-46c0-9a50-4e74c53e1f30 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:32.398238 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3f1ad-9b8a-4df3-b842-48247c77a071 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:32.398864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3f1ad-9b8a-4df3-b842-48247c77a071 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:32.399371 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3f1ad-9b8a-4df3-b842-48247c77a071 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:32.405680 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 124/246] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:32.406429 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:32.408793 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89195c64-f185-4318-9ca0-78a7d766378a None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:32.409313 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89195c64-f185-4318-9ca0-78a7d766378a None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:32.409785 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89195c64-f185-4318-9ca0-78a7d766378a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:32.421610 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 123/247] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:34:32.431708 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980b088-d2fa-46c0-9a50-4e74c53e1f30 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:32.443755 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-89195c64-f185-4318-9ca0-78a7d766378a None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:32.445348 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-89195c64-f185-4318-9ca0-78a7d766378a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:32.529453 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 125/248] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:32.530123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89195c64-f185-4318-9ca0-78a7d766378a None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:32.552210 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980b088-d2fa-46c0-9a50-4e74c53e1f30 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:32.602697 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c679a-fc60-4131-ab26-7ed1762595a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:32.603347 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c679a-fc60-4131-ab26-7ed1762595a6 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:32.603623 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c679a-fc60-4131-ab26-7ed1762595a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:32.603897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c679a-fc60-4131-ab26-7ed1762595a6 None admin] PATH_INFO: `/v3/projects/22414db73cef49549c4930bec9f6b5dc` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:32.606665 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c679a-fc60-4131-ab26-7ed1762595a6 None admin] RBAC: Authorizing `identity:get_project(project_id=22414db73cef49549c4930bec9f6b5dc)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:32.608309 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c679a-fc60-4131-ab26-7ed1762595a6 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:32.610626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 124/249] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:32 2022] GET /identity/v3/projects/22414db73cef49549c4930bec9f6b5dc => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:32.617678 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c679a-fc60-4131-ab26-7ed1762595a6 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:32.628840 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:32.672024 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89195c64-f185-4318-9ca0-78a7d766378a None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:32.691840 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:32.717106 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5937ef4-9641-44d2-87b5-b5a506829a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:32.718099 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5937ef4-9641-44d2-87b5-b5a506829a7b None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:32.718578 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5937ef4-9641-44d2-87b5-b5a506829a7b None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:32.719009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5937ef4-9641-44d2-87b5-b5a506829a7b None admin] PATH_INFO: `/v3/roles/member` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:32.733525 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5937ef4-9641-44d2-87b5-b5a506829a7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:32.739673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-e5937ef4-9641-44d2-87b5-b5a506829a7b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:32.739673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:32.739673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:32.739673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:32.739673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:32.739673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:32.739673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:32.739673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:32.739673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:32.739673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:32.739673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:32.739673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:32.739673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:32.739673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:32.739673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:32.739673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:32.740602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:32.740602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:32.740602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:32.740602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:32.740602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:32.740602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:32.740602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:32.740602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:32.740602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:32.740602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:32.740602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:32.740602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:32.740602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:32.740602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:32.740602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:32.740602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:32.741816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:32.741816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:32.741816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:32.741816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:32.741816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:32.741816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:32.741816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:32.741816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:32.741816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:32.741816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:32.741816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:32.741816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:34:32.741816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 126/250] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:32.742474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fedd0d-8bca-4e20-9a5f-404decb1d20f None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:32.742627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fedd0d-8bca-4e20-9a5f-404decb1d20f None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:32.742900 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fedd0d-8bca-4e20-9a5f-404decb1d20f None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:32.744241 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 127/251] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:32.750186 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fedd0d-8bca-4e20-9a5f-404decb1d20f None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:32.761477 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c679a-fc60-4131-ab26-7ed1762595a6 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:32.816389 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd291e-2066-41c7-86f8-c3ac7b1aafac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:32.817026 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd291e-2066-41c7-86f8-c3ac7b1aafac None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:32.817307 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd291e-2066-41c7-86f8-c3ac7b1aafac None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:32.817577 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd291e-2066-41c7-86f8-c3ac7b1aafac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:32.818334 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd291e-2066-41c7-86f8-c3ac7b1aafac None admin] RBAC: Adding query filter params (role.id=b68669d0f20b4ac8a1a7d46142b03424, scope.project.id=22414db73cef49549c4930bec9f6b5dc, user.id=548e7a814507494bb420b223caffaaab) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:32.818600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd291e-2066-41c7-86f8-c3ac7b1aafac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:32.820036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd291e-2066-41c7-86f8-c3ac7b1aafac None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:32.826410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 125/252] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:34:32 2022] GET /identity/v3/role_assignments?user.id=548e7a814507494bb420b223caffaaab&scope.project.id=22414db73cef49549c4930bec9f6b5dc&role.id=b68669d0f20b4ac8a1a7d46142b03424 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:32.828369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3ab4c-08d5-433a-b830-4f31a2a6d01d None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:32.828682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3ab4c-08d5-433a-b830-4f31a2a6d01d None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:32.828954 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3ab4c-08d5-433a-b830-4f31a2a6d01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:32.850439 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-bdc3ab4c-08d5-433a-b830-4f31a2a6d01d None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:32.851653 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-bdc3ab4c-08d5-433a-b830-4f31a2a6d01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:32.897840 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fedd0d-8bca-4e20-9a5f-404decb1d20f None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:32.916775 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:32.937576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce40cb-469e-4f20-8037-35c0a0cccfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:32.938246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce40cb-469e-4f20-8037-35c0a0cccfa3 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:32.938718 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce40cb-469e-4f20-8037-35c0a0cccfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:32.939390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce40cb-469e-4f20-8037-35c0a0cccfa3 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:32.940086 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce40cb-469e-4f20-8037-35c0a0cccfa3 None admin] RBAC: Adding query filter params (name=member) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:32.940383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce40cb-469e-4f20-8037-35c0a0cccfa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:32.941875 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce40cb-469e-4f20-8037-35c0a0cccfa3 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:32.949175 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 126/253] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:32.950874 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc3ab4c-08d5-433a-b830-4f31a2a6d01d None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:32.956328 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 128/254] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:34:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:32.966942 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce40cb-469e-4f20-8037-35c0a0cccfa3 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:33.065009 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:33.154725 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce40cb-469e-4f20-8037-35c0a0cccfa3 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:33.163503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc3ab4c-08d5-433a-b830-4f31a2a6d01d None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:33.227226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52e148-ebcc-47a5-b606-fc99f9484d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:33.227871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52e148-ebcc-47a5-b606-fc99f9484d5b None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:33.228148 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52e148-ebcc-47a5-b606-fc99f9484d5b None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:33.228423 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52e148-ebcc-47a5-b606-fc99f9484d5b None admin] PATH_INFO: `/v3/users/a607948d7f414d0caf2eaee9eb0ec330` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:33.233161 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52e148-ebcc-47a5-b606-fc99f9484d5b None admin] RBAC: Authorizing `identity:get_user(user_id=a607948d7f414d0caf2eaee9eb0ec330)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:33.234620 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52e148-ebcc-47a5-b606-fc99f9484d5b None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:33.236817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 129/255] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:32 2022] GET /identity/v3/users/a607948d7f414d0caf2eaee9eb0ec330 => generated 297 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:33.237556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdcbf1-44f4-4fff-8ebb-ccbcebce2dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:33.238276 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdcbf1-44f4-4fff-8ebb-ccbcebce2dd7 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:33.238558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdcbf1-44f4-4fff-8ebb-ccbcebce2dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:33.238833 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdcbf1-44f4-4fff-8ebb-ccbcebce2dd7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:33.242206 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52e148-ebcc-47a5-b606-fc99f9484d5b None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:33.263253 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cdcbf1-44f4-4fff-8ebb-ccbcebce2dd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:33.268332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-07cdcbf1-44f4-4fff-8ebb-ccbcebce2dd7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:34:33.268332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:33.268332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:33.268332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:33.268332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:33.268332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:33.268332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:33.268332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:33.268332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:33.268332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:33.268332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:33.268332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:33.268332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:33.268332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:33.268332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:33.268332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:33.269259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:33.269259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:33.269259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:33.269259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:33.269259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:33.269259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:33.269259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:33.269259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:33.269259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:33.269259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:33.269259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:33.269259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:33.269259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:33.269259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:33.269259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:33.269259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:33.270060 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:33.270060 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:33.270060 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:33.270060 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:33.270060 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:33.270060 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:33.270060 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:33.270060 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:33.270060 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:33.270060 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:33.270060 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:34:33.270060 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:34:33.270060 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 127/256] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:34:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:33.277487 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdcbf1-44f4-4fff-8ebb-ccbcebce2dd7 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:33.316878 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:33.371604 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52e148-ebcc-47a5-b606-fc99f9484d5b None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:33.412370 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418df791-598b-4e11-87f5-148272e54a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:33.413073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418df791-598b-4e11-87f5-148272e54a24 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:33.413356 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418df791-598b-4e11-87f5-148272e54a24 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:33.413636 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418df791-598b-4e11-87f5-148272e54a24 None admin] PATH_INFO: `/v3/projects/22414db73cef49549c4930bec9f6b5dc` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:33.414773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418df791-598b-4e11-87f5-148272e54a24 None admin] RBAC: Authorizing `identity:get_project(project_id=22414db73cef49549c4930bec9f6b5dc)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:33.416248 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418df791-598b-4e11-87f5-148272e54a24 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:33.419591 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 130/257] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:33 2022] GET /identity/v3/projects/22414db73cef49549c4930bec9f6b5dc => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:33.420358 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418df791-598b-4e11-87f5-148272e54a24 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:33.517497 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdcbf1-44f4-4fff-8ebb-ccbcebce2dd7 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:33.536858 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:33.554431 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a932f-4e49-48c4-ae00-501e54e449c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:33.555285 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a932f-4e49-48c4-ae00-501e54e449c9 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:33.555728 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a932f-4e49-48c4-ae00-501e54e449c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:33.556987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a932f-4e49-48c4-ae00-501e54e449c9 None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:33.556987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51a932f-4e49-48c4-ae00-501e54e449c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:33.557501 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51a932f-4e49-48c4-ae00-501e54e449c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:33.559049 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51a932f-4e49-48c4-ae00-501e54e449c9 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:33.567723 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 128/258] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:33.568774 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a932f-4e49-48c4-ae00-501e54e449c9 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:33.654631 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418df791-598b-4e11-87f5-148272e54a24 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:33.673959 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a932f-4e49-48c4-ae00-501e54e449c9 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:33.699776 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:33.708324 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996edd6d-ec5a-42c6-ae88-d843554cc656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:33.709264 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996edd6d-ec5a-42c6-ae88-d843554cc656 None admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:33.709612 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996edd6d-ec5a-42c6-ae88-d843554cc656 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:33.709948 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996edd6d-ec5a-42c6-ae88-d843554cc656 None admin] PATH_INFO: `/v3/projects/22414db73cef49549c4930bec9f6b5dc/users/a607948d7f414d0caf2eaee9eb0ec330/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:33.711985 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996edd6d-ec5a-42c6-ae88-d843554cc656 None admin] RBAC: Authorizing `identity:create_grant(project_id=22414db73cef49549c4930bec9f6b5dc, user_id=a607948d7f414d0caf2eaee9eb0ec330, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:33.713519 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996edd6d-ec5a-42c6-ae88-d843554cc656 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:33.731415 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006dd15c-2a90-4af0-afe0-89e6f8fc93da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:33.732178 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006dd15c-2a90-4af0-afe0-89e6f8fc93da None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:33.732590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006dd15c-2a90-4af0-afe0-89e6f8fc93da None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:33.732932 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006dd15c-2a90-4af0-afe0-89e6f8fc93da None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:33.744075 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006dd15c-2a90-4af0-afe0-89e6f8fc93da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:33.749079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-006dd15c-2a90-4af0-afe0-89e6f8fc93da None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 00:34:33.749079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:33.749079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:33.749079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:33.749079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:33.749079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:33.749079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:33.749079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:33.749079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:33.749079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:33.749079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:33.749079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:33.749079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:33.749079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:33.749079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:33.749079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:33.750269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:33.750269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:33.750269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:33.750269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:33.750269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:33.750269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:33.750269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:33.750269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:33.750269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:33.750269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:33.750269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:33.750269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:33.750269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:33.750269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:33.750269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:33.750269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:33.751845 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:33.751845 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:33.751845 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:33.751845 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:33.751845 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:33.751845 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:33.751845 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:33.751845 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:33.751845 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:33.751845 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:33.751845 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 00:34:33.751845 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:34:33.752993 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 131/259] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Aug 26 00:34:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:33.753481 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006dd15c-2a90-4af0-afe0-89e6f8fc93da None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:33.792163 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 129/260] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Aug 26 00:34:33 2022] PUT /identity/v3/projects/22414db73cef49549c4930bec9f6b5dc/users/a607948d7f414d0caf2eaee9eb0ec330/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:34:33.793057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996edd6d-ec5a-42c6-ae88-d843554cc656 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:33.853339 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:33.940121 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006dd15c-2a90-4af0-afe0-89e6f8fc93da None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:33.969008 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996edd6d-ec5a-42c6-ae88-d843554cc656 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:34.002493 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae346de-d24a-4d1a-aa02-a862e110aee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:34.008236 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae346de-d24a-4d1a-aa02-a862e110aee5 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:34.008236 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae346de-d24a-4d1a-aa02-a862e110aee5 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:34.008236 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae346de-d24a-4d1a-aa02-a862e110aee5 None admin] PATH_INFO: `/v3/users/0771c65623834672a441c17da5cc4476` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:34.008236 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae346de-d24a-4d1a-aa02-a862e110aee5 None admin] RBAC: Authorizing `identity:get_user(user_id=0771c65623834672a441c17da5cc4476)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:34.008236 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae346de-d24a-4d1a-aa02-a862e110aee5 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:34.008236 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 132/261] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:33 2022] GET /identity/v3/users/0771c65623834672a441c17da5cc4476 => generated 283 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:34.009220 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99028164-d27c-4e83-9784-66a7019184c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:34.009875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99028164-d27c-4e83-9784-66a7019184c6 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:34.010109 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99028164-d27c-4e83-9784-66a7019184c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:34.010378 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99028164-d27c-4e83-9784-66a7019184c6 None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:34.011052 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99028164-d27c-4e83-9784-66a7019184c6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:34.011350 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99028164-d27c-4e83-9784-66a7019184c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:34.012761 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99028164-d27c-4e83-9784-66a7019184c6 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:34.016045 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae346de-d24a-4d1a-aa02-a862e110aee5 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:34.032068 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 130/262] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:34:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:34.037892 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99028164-d27c-4e83-9784-66a7019184c6 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:34.100928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:34.145425 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:34.156515 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99028164-d27c-4e83-9784-66a7019184c6 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:34.175547 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae346de-d24a-4d1a-aa02-a862e110aee5 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:34.192930 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627711a-53ef-43c1-9dfe-59062d1fb23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:34.193494 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627711a-53ef-43c1-9dfe-59062d1fb23e None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:34.193770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627711a-53ef-43c1-9dfe-59062d1fb23e None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:34.194041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627711a-53ef-43c1-9dfe-59062d1fb23e None admin] PATH_INFO: `/v3/users/0771c65623834672a441c17da5cc4476` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:34.195178 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c627711a-53ef-43c1-9dfe-59062d1fb23e None admin] RBAC: Authorizing `identity:get_user(user_id=0771c65623834672a441c17da5cc4476)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:34.196637 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c627711a-53ef-43c1-9dfe-59062d1fb23e None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:34.198447 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 131/263] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:34 2022] GET /identity/v3/users/0771c65623834672a441c17da5cc4476 => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:34.205537 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627711a-53ef-43c1-9dfe-59062d1fb23e None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:34.244753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b5fca-f19c-48ec-bd08-c632a35bbd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:34.245429 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b5fca-f19c-48ec-bd08-c632a35bbd1d None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:34.245756 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b5fca-f19c-48ec-bd08-c632a35bbd1d None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:34.246032 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b5fca-f19c-48ec-bd08-c632a35bbd1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:34.328272 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979b5fca-f19c-48ec-bd08-c632a35bbd1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:34.330384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979b5fca-f19c-48ec-bd08-c632a35bbd1d None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:34.397866 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:34.454361 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627711a-53ef-43c1-9dfe-59062d1fb23e None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:34.490905 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 133/264] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:34:34.496561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b5fca-f19c-48ec-bd08-c632a35bbd1d None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:34.617215 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa248c44-eb4c-4e42-ab93-32a7e91dca85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:34.617847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa248c44-eb4c-4e42-ab93-32a7e91dca85 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:34.618129 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa248c44-eb4c-4e42-ab93-32a7e91dca85 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:34.618405 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa248c44-eb4c-4e42-ab93-32a7e91dca85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:34.643024 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b5fca-f19c-48ec-bd08-c632a35bbd1d None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:34.690205 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d28cc7-3f15-460b-929a-c31118090e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:34.690838 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d28cc7-3f15-460b-929a-c31118090e93 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:34.691110 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d28cc7-3f15-460b-929a-c31118090e93 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:34.691376 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d28cc7-3f15-460b-929a-c31118090e93 None admin] PATH_INFO: `/v3/projects/22414db73cef49549c4930bec9f6b5dc` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:34.692908 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d28cc7-3f15-460b-929a-c31118090e93 None admin] RBAC: Authorizing `identity:get_project(project_id=22414db73cef49549c4930bec9f6b5dc)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:34.694757 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d28cc7-3f15-460b-929a-c31118090e93 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:34.697202 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 134/265] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:34 2022] GET /identity/v3/projects/22414db73cef49549c4930bec9f6b5dc => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:34.704177 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:34.705589 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d28cc7-3f15-460b-929a-c31118090e93 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:34.749050 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:34.807536 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa248c44-eb4c-4e42-ab93-32a7e91dca85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:34.809091 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa248c44-eb4c-4e42-ab93-32a7e91dca85 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:34.942038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d28cc7-3f15-460b-929a-c31118090e93 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:35.033733 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94278bdc-146f-482d-923d-fe79326c4ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:35.034281 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94278bdc-146f-482d-923d-fe79326c4ba8 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:35.034565 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94278bdc-146f-482d-923d-fe79326c4ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:35.034840 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94278bdc-146f-482d-923d-fe79326c4ba8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:35.035585 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94278bdc-146f-482d-923d-fe79326c4ba8 None admin] RBAC: Adding query filter params (role.id=1a33e3c8cb8347feb9498595886cc621, scope.project.id=22414db73cef49549c4930bec9f6b5dc, user.id=0771c65623834672a441c17da5cc4476) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:35.035897 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94278bdc-146f-482d-923d-fe79326c4ba8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:35.043026 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 132/266] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 838 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:34:35.044134 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94278bdc-146f-482d-923d-fe79326c4ba8 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:35.049018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa248c44-eb4c-4e42-ab93-32a7e91dca85 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:35.050811 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 135/267] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:34:34 2022] GET /identity/v3/role_assignments?user.id=0771c65623834672a441c17da5cc4476&scope.project.id=22414db73cef49549c4930bec9f6b5dc&role.id=1a33e3c8cb8347feb9498595886cc621 => generated 608 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:35.070362 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9249f47-4b07-43ac-84a5-0ab699fce4df None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:35.070656 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9249f47-4b07-43ac-84a5-0ab699fce4df None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:35.070936 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9249f47-4b07-43ac-84a5-0ab699fce4df None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:35.072581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 136/268] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:34:35.077961 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b59b2-a11d-45d3-a9f2-65680f8de4d4 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:35.082679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b59b2-a11d-45d3-a9f2-65680f8de4d4 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:35.082679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b59b2-a11d-45d3-a9f2-65680f8de4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:35.114069 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-f10b59b2-a11d-45d3-a9f2-65680f8de4d4 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:35.115569 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f10b59b2-a11d-45d3-a9f2-65680f8de4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:35.175463 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:35.214760 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 137/269] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:35.251875 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf8b7c-21f4-4c46-bddf-3d40d92afa17 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:35.252199 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf8b7c-21f4-4c46-bddf-3d40d92afa17 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:35.252436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf8b7c-21f4-4c46-bddf-3d40d92afa17 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:35.254156 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 138/270] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:35.259407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2157325-69da-4c69-8cb9-c71565fe2a0e None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:35.259699 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2157325-69da-4c69-8cb9-c71565fe2a0e None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:35.259986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2157325-69da-4c69-8cb9-c71565fe2a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:35.274179 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa248c44-eb4c-4e42-ab93-32a7e91dca85 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:35.288122 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:35.293641 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-f2157325-69da-4c69-8cb9-c71565fe2a0e None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:35.294867 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f2157325-69da-4c69-8cb9-c71565fe2a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:35.361007 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f96a6-dc14-4d3b-a849-224ab8b23639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:35.361652 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f96a6-dc14-4d3b-a849-224ab8b23639 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:35.361934 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f96a6-dc14-4d3b-a849-224ab8b23639 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:35.362211 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f96a6-dc14-4d3b-a849-224ab8b23639 None admin] PATH_INFO: `/v3/projects/22414db73cef49549c4930bec9f6b5dc` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:35.363548 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f96a6-dc14-4d3b-a849-224ab8b23639 None admin] RBAC: Authorizing `identity:get_project(project_id=22414db73cef49549c4930bec9f6b5dc)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:35.366631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f96a6-dc14-4d3b-a849-224ab8b23639 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:35.370716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 133/271] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:35 2022] GET /identity/v3/projects/22414db73cef49549c4930bec9f6b5dc => generated 302 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:35.377013 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f96a6-dc14-4d3b-a849-224ab8b23639 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:35.414785 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 139/272] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:34:35.532139 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2157325-69da-4c69-8cb9-c71565fe2a0e None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:35.544188 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f96a6-dc14-4d3b-a849-224ab8b23639 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:35.590608 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:35.602125 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c4da2-a08e-41c1-ba72-6ecfec071f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:35.602746 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c4da2-a08e-41c1-ba72-6ecfec071f56 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:35.608926 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c4da2-a08e-41c1-ba72-6ecfec071f56 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:35.608926 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c4da2-a08e-41c1-ba72-6ecfec071f56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:35.608926 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c4da2-a08e-41c1-ba72-6ecfec071f56 None admin] RBAC: Adding query filter params (role.id=423aa6ac8b05402da7587e83e5c4ce09, scope.project.id=22414db73cef49549c4930bec9f6b5dc, user.id=0771c65623834672a441c17da5cc4476) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:35.608926 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c4da2-a08e-41c1-ba72-6ecfec071f56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:35.608926 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c4da2-a08e-41c1-ba72-6ecfec071f56 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:35.617407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 134/273] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:34:35 2022] GET /identity/v3/role_assignments?user.id=0771c65623834672a441c17da5cc4476&scope.project.id=22414db73cef49549c4930bec9f6b5dc&role.id=423aa6ac8b05402da7587e83e5c4ce09 => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:35.660514 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2157325-69da-4c69-8cb9-c71565fe2a0e None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:35.702482 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec283dc2-85ba-49e8-8118-a92fdb1e951b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:35.703136 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec283dc2-85ba-49e8-8118-a92fdb1e951b None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:35.703414 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec283dc2-85ba-49e8-8118-a92fdb1e951b None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:35.703687 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec283dc2-85ba-49e8-8118-a92fdb1e951b None admin] PATH_INFO: `/v3/domains/default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:35.705388 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec283dc2-85ba-49e8-8118-a92fdb1e951b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:35.706852 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec283dc2-85ba-49e8-8118-a92fdb1e951b None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:35.708622 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 140/274] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:34:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:35.714179 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c4da2-a08e-41c1-ba72-6ecfec071f56 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:35.804638 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec283dc2-85ba-49e8-8118-a92fdb1e951b None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:35.815838 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:35.862297 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c4da2-a08e-41c1-ba72-6ecfec071f56 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:35.904957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97291f6-f334-4bbf-9438-e08b2f916bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:35.905586 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97291f6-f334-4bbf-9438-e08b2f916bb2 None admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:35.905870 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97291f6-f334-4bbf-9438-e08b2f916bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:35.906144 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97291f6-f334-4bbf-9438-e08b2f916bb2 None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:35.906811 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97291f6-f334-4bbf-9438-e08b2f916bb2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:35.908839 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97291f6-f334-4bbf-9438-e08b2f916bb2 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:35.928447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec283dc2-85ba-49e8-8118-a92fdb1e951b None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:35.965701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36d4a1-0f5e-4eea-a5ae-c362ea01ab81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:35.966402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36d4a1-0f5e-4eea-a5ae-c362ea01ab81 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:35.966637 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36d4a1-0f5e-4eea-a5ae-c362ea01ab81 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:35.966880 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36d4a1-0f5e-4eea-a5ae-c362ea01ab81 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:35.977648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36d4a1-0f5e-4eea-a5ae-c362ea01ab81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:35.983927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-9d36d4a1-0f5e-4eea-a5ae-c362ea01ab81 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:35.983927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:35.983927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:35.983927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:35.983927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:35.983927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:35.983927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:35.983927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:35.983927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:35.983927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:35.983927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:35.983927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:35.983927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:35.983927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:35.983927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:35.983927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:35.987276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:35.987276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:35.987276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:35.987276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:35.987276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:35.987276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:35.987276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:35.987276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:35.987276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:35.987276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:35.987276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:35.987276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:35.987276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:35.987276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:35.987276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:35.987276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:35.988112 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:35.988112 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:35.988112 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:35.988112 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:35.988112 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:35.988112 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:35.988112 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:35.988112 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:35.988112 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:35.988112 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:35.988112 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:35.988112 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:34:35.988112 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 141/275] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:35.988112 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f183aa-662a-4114-b964-3cbe53af5902 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:35.989313 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f183aa-662a-4114-b964-3cbe53af5902 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:35.989313 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f183aa-662a-4114-b964-3cbe53af5902 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:35.989313 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 142/276] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:35.991616 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f183aa-662a-4114-b964-3cbe53af5902 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:36.014831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 135/277] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Aug 26 00:34:35 2022] POST /identity/v3/users => generated 293 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:34:36.016681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78b2cd-e6a0-4aac-af9b-6d0de939d1f1 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:36.017154 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78b2cd-e6a0-4aac-af9b-6d0de939d1f1 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:36.018058 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78b2cd-e6a0-4aac-af9b-6d0de939d1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:36.042463 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-5b78b2cd-e6a0-4aac-af9b-6d0de939d1f1 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:36.044188 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-5b78b2cd-e6a0-4aac-af9b-6d0de939d1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:36.064907 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:36.113077 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:36.118369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f183aa-662a-4114-b964-3cbe53af5902 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:36.155503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 136/278] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:36.171790 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3adfe2e-b27d-49ef-86a9-f20fec0f245f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:36.172780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3adfe2e-b27d-49ef-86a9-f20fec0f245f None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:36.173266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3adfe2e-b27d-49ef-86a9-f20fec0f245f None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:36.173776 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3adfe2e-b27d-49ef-86a9-f20fec0f245f None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:36.174452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3adfe2e-b27d-49ef-86a9-f20fec0f245f None admin] RBAC: Adding query filter params (name=member) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:36.174810 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3adfe2e-b27d-49ef-86a9-f20fec0f245f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:36.176191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3adfe2e-b27d-49ef-86a9-f20fec0f245f None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:36.185272 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 143/279] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:34:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:36.191119 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78b2cd-e6a0-4aac-af9b-6d0de939d1f1 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:36.295138 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78b2cd-e6a0-4aac-af9b-6d0de939d1f1 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:36.332058 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d01d96-3548-4491-bd95-ddf17a77f8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:36.334008 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d01d96-3548-4491-bd95-ddf17a77f8a0 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:36.334356 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d01d96-3548-4491-bd95-ddf17a77f8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:36.334644 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d01d96-3548-4491-bd95-ddf17a77f8a0 None admin] PATH_INFO: `/v3/users/8bae51e1801d4f70b24f2fdf30c93ddd` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:36.335107 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4f797-b0c1-4873-8a0b-843de838b0ae None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:36.335580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4f797-b0c1-4873-8a0b-843de838b0ae None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:36.336027 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4f797-b0c1-4873-8a0b-843de838b0ae None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:36.338881 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 144/280] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:34:36.347132 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37c372-da4c-4234-bdb5-4b4eef110106 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:36.347132 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37c372-da4c-4234-bdb5-4b4eef110106 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:36.347345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37c372-da4c-4234-bdb5-4b4eef110106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:36.357912 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d01d96-3548-4491-bd95-ddf17a77f8a0 None admin] RBAC: Authorizing `identity:get_user(user_id=8bae51e1801d4f70b24f2fdf30c93ddd)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:36.359376 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d01d96-3548-4491-bd95-ddf17a77f8a0 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:36.361903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 137/281] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:36 2022] GET /identity/v3/users/8bae51e1801d4f70b24f2fdf30c93ddd => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:36.367393 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d01d96-3548-4491-bd95-ddf17a77f8a0 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:36.373555 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-ce37c372-da4c-4234-bdb5-4b4eef110106 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:36.375008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-ce37c372-da4c-4234-bdb5-4b4eef110106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:36.410713 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:36.466693 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 145/282] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:34:36.467884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37c372-da4c-4234-bdb5-4b4eef110106 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:36.492798 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d01d96-3548-4491-bd95-ddf17a77f8a0 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:36.531812 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5980b-375c-4578-9801-17cbcb77b7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:36.532889 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5980b-375c-4578-9801-17cbcb77b7ce None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:36.533174 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5980b-375c-4578-9801-17cbcb77b7ce None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:36.533447 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5980b-375c-4578-9801-17cbcb77b7ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:36.534186 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5980b-375c-4578-9801-17cbcb77b7ce None admin] RBAC: Adding query filter params (role.id=5032636d8729461c9926a59b9bcd041b, user.id=8bae51e1801d4f70b24f2fdf30c93ddd, scope.system=all) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:36.534494 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5980b-375c-4578-9801-17cbcb77b7ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:36.536534 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5980b-375c-4578-9801-17cbcb77b7ce None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:36.538643 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:36.545002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 138/283] 10.222.0.59 () {56 vars in 1308 bytes} [Fri Aug 26 00:34:36 2022] GET /identity/v3/role_assignments?user.id=8bae51e1801d4f70b24f2fdf30c93ddd&scope.system=all&role.id=5032636d8729461c9926a59b9bcd041b => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:36.595981 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37c372-da4c-4234-bdb5-4b4eef110106 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:36.636102 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b77c65-ea10-4fb3-9be5-d6a0cbb673d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:36.636838 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b77c65-ea10-4fb3-9be5-d6a0cbb673d5 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:36.637124 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b77c65-ea10-4fb3-9be5-d6a0cbb673d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:36.637401 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b77c65-ea10-4fb3-9be5-d6a0cbb673d5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:36.649945 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b77c65-ea10-4fb3-9be5-d6a0cbb673d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:36.655319 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-69b77c65-ea10-4fb3-9be5-d6a0cbb673d5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 00:34:36.655319 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:36.655319 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:36.655319 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:36.655319 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:36.655319 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:36.655319 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:36.655319 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:36.655319 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:36.655319 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:36.655319 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:36.655319 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:36.655319 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:36.655319 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:36.655319 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:36.655319 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:36.656277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:36.656277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:36.656277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:36.656277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:36.656277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:36.656277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:36.656277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:36.656277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:36.656277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:36.656277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:36.656277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:36.656277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:36.656277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:36.656277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:36.656277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:36.656277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:36.657115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:36.657115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:36.657115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:36.657115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:36.657115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:36.657115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:36.657115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:36.657115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:36.657115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:36.657115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:36.657115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 00:34:36.657115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:34:36.657115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 146/284] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:36.663162 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5980b-375c-4578-9801-17cbcb77b7ce None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:36.786096 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:36.790025 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5980b-375c-4578-9801-17cbcb77b7ce None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:36.830938 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b77c65-ea10-4fb3-9be5-d6a0cbb673d5 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:36.839645 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b9109-65cd-4449-867e-fc88051bef90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:36.840255 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b9109-65cd-4449-867e-fc88051bef90 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:36.840574 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b9109-65cd-4449-867e-fc88051bef90 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:36.840825 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b9109-65cd-4449-867e-fc88051bef90 None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:36.841486 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b9109-65cd-4449-867e-fc88051bef90 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:36.841787 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b9109-65cd-4449-867e-fc88051bef90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:36.843145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b9109-65cd-4449-867e-fc88051bef90 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:36.859188 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 139/285] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:34:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:36.865283 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b9109-65cd-4449-867e-fc88051bef90 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:37.029704 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b77c65-ea10-4fb3-9be5-d6a0cbb673d5 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:37.031754 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b9109-65cd-4449-867e-fc88051bef90 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:37.065811 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:37.107664 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d5fa6b-cceb-4b49-a0c4-21a5178d9123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:37.108334 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5fa6b-cceb-4b49-a0c4-21a5178d9123 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:37.108618 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5fa6b-cceb-4b49-a0c4-21a5178d9123 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:37.108898 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5fa6b-cceb-4b49-a0c4-21a5178d9123 None admin] PATH_INFO: `/v3/users/548e7a814507494bb420b223caffaaab` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:37.113291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d5fa6b-cceb-4b49-a0c4-21a5178d9123 None admin] RBAC: Authorizing `identity:get_user(user_id=548e7a814507494bb420b223caffaaab)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:37.114678 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d5fa6b-cceb-4b49-a0c4-21a5178d9123 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:37.116320 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55acf8-96aa-428b-9d07-e6d3708b17aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:37.116989 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55acf8-96aa-428b-9d07-e6d3708b17aa None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:37.117263 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55acf8-96aa-428b-9d07-e6d3708b17aa None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:37.117531 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55acf8-96aa-428b-9d07-e6d3708b17aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:37.118703 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 140/286] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:36 2022] GET /identity/v3/users/548e7a814507494bb420b223caffaaab => generated 297 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:37.125155 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d5fa6b-cceb-4b49-a0c4-21a5178d9123 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:37.139782 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a55acf8-96aa-428b-9d07-e6d3708b17aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:37.145126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-7a55acf8-96aa-428b-9d07-e6d3708b17aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:37.145126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:37.145126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:37.145126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:37.145126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:37.145126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:37.145126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:37.145126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:37.145126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:37.145126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:37.145126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:37.145126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:37.145126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:37.145126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:37.145126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:37.145126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:37.146020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:37.146020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:37.146020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:37.146020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:37.146020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:37.146020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:37.146020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:37.146020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:37.146020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:37.146020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:37.146020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:37.146020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:37.146020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:37.146020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:37.146020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:37.146020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:37.146824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:37.146824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:37.146824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:37.146824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:37.146824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:37.146824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:37.146824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:37.146824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:37.146824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:37.146824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:37.146824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:37.146824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:34:37.147884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 147/287] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:37.154802 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55acf8-96aa-428b-9d07-e6d3708b17aa None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:37.174793 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:37.303275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d5fa6b-cceb-4b49-a0c4-21a5178d9123 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:37.325309 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55acf8-96aa-428b-9d07-e6d3708b17aa None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:37.374102 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059d235-8570-4894-a707-d826ac1ee38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:37.374975 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059d235-8570-4894-a707-d826ac1ee38a None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:37.375444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059d235-8570-4894-a707-d826ac1ee38a None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:37.375905 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059d235-8570-4894-a707-d826ac1ee38a None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:37.377172 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059d235-8570-4894-a707-d826ac1ee38a None admin] RBAC: Adding query filter params (name=member) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:37.377698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059d235-8570-4894-a707-d826ac1ee38a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:37.380220 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059d235-8570-4894-a707-d826ac1ee38a None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:37.386016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445b2ea-0f8e-420a-831e-9cee7a9456b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:37.386681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445b2ea-0f8e-420a-831e-9cee7a9456b4 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:37.386964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445b2ea-0f8e-420a-831e-9cee7a9456b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:37.387236 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445b2ea-0f8e-420a-831e-9cee7a9456b4 None admin] PATH_INFO: `/v3/projects/22414db73cef49549c4930bec9f6b5dc` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:37.388399 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2445b2ea-0f8e-420a-831e-9cee7a9456b4 None admin] RBAC: Authorizing `identity:get_project(project_id=22414db73cef49549c4930bec9f6b5dc)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:37.389866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2445b2ea-0f8e-420a-831e-9cee7a9456b4 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:37.391609 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 141/288] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:37 2022] GET /identity/v3/projects/22414db73cef49549c4930bec9f6b5dc => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:37.404892 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445b2ea-0f8e-420a-831e-9cee7a9456b4 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:37.408061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 148/289] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:34:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:37.413807 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059d235-8570-4894-a707-d826ac1ee38a None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:37.473852 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:37.567488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059d235-8570-4894-a707-d826ac1ee38a None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:37.599449 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:37.607209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41c269-b1cc-4a62-a836-9af92eac95ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:37.607860 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41c269-b1cc-4a62-a836-9af92eac95ce None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:37.608138 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41c269-b1cc-4a62-a836-9af92eac95ce None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:37.608414 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41c269-b1cc-4a62-a836-9af92eac95ce None admin] PATH_INFO: `/v3/users/a607948d7f414d0caf2eaee9eb0ec330` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:37.614400 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445b2ea-0f8e-420a-831e-9cee7a9456b4 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:37.614508 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e41c269-b1cc-4a62-a836-9af92eac95ce None admin] RBAC: Authorizing `identity:get_user(user_id=a607948d7f414d0caf2eaee9eb0ec330)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:37.614508 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e41c269-b1cc-4a62-a836-9af92eac95ce None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:37.615854 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 149/290] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:37 2022] GET /identity/v3/users/a607948d7f414d0caf2eaee9eb0ec330 => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:37.625818 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41c269-b1cc-4a62-a836-9af92eac95ce None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:37.684326 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d9573-b508-4ab8-9bc4-37e120cae136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:37.685077 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d9573-b508-4ab8-9bc4-37e120cae136 None admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:37.685356 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d9573-b508-4ab8-9bc4-37e120cae136 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:37.685671 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d9573-b508-4ab8-9bc4-37e120cae136 None admin] PATH_INFO: `/v3/projects/22414db73cef49549c4930bec9f6b5dc/users/548e7a814507494bb420b223caffaaab/roles/b68669d0f20b4ac8a1a7d46142b03424` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:37.689981 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056d9573-b508-4ab8-9bc4-37e120cae136 None admin] RBAC: Authorizing `identity:create_grant(project_id=22414db73cef49549c4930bec9f6b5dc, user_id=548e7a814507494bb420b223caffaaab, role_id=b68669d0f20b4ac8a1a7d46142b03424)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:37.693659 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056d9573-b508-4ab8-9bc4-37e120cae136 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:37.770729 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 142/291] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Aug 26 00:34:37 2022] PUT /identity/v3/projects/22414db73cef49549c4930bec9f6b5dc/users/548e7a814507494bb420b223caffaaab/roles/b68669d0f20b4ac8a1a7d46142b03424 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:34:37.785053 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41c269-b1cc-4a62-a836-9af92eac95ce None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:37.800099 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:37.871076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977dbff-5174-44cb-a6f0-366ec232d335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:37.871719 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977dbff-5174-44cb-a6f0-366ec232d335 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:37.871999 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977dbff-5174-44cb-a6f0-366ec232d335 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:37.872279 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977dbff-5174-44cb-a6f0-366ec232d335 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:37.967515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b977dbff-5174-44cb-a6f0-366ec232d335 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:37.969010 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b977dbff-5174-44cb-a6f0-366ec232d335 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:38.084271 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 150/292] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:34:38.090077 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d9573-b508-4ab8-9bc4-37e120cae136 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:38.214870 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d9573-b508-4ab8-9bc4-37e120cae136 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:38.249364 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c7019-370f-4902-80a7-04fbb4a062c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:38.249921 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c7019-370f-4902-80a7-04fbb4a062c0 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:38.250204 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c7019-370f-4902-80a7-04fbb4a062c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:38.250477 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c7019-370f-4902-80a7-04fbb4a062c0 None admin] PATH_INFO: `/v3/projects/22414db73cef49549c4930bec9f6b5dc` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:38.252151 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c7019-370f-4902-80a7-04fbb4a062c0 None admin] RBAC: Authorizing `identity:get_project(project_id=22414db73cef49549c4930bec9f6b5dc)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:38.253700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c7019-370f-4902-80a7-04fbb4a062c0 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:38.255656 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 143/293] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:38 2022] GET /identity/v3/projects/22414db73cef49549c4930bec9f6b5dc => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:38.261624 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977dbff-5174-44cb-a6f0-366ec232d335 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:38.265949 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:38.387011 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977dbff-5174-44cb-a6f0-366ec232d335 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:38.428207 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08336178-c0b4-47f9-9798-f441a7ac9ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:38.428859 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08336178-c0b4-47f9-9798-f441a7ac9ca7 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:38.429193 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08336178-c0b4-47f9-9798-f441a7ac9ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:38.429469 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08336178-c0b4-47f9-9798-f441a7ac9ca7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:38.430241 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08336178-c0b4-47f9-9798-f441a7ac9ca7 None admin] RBAC: Adding query filter params (role.id=5032636d8729461c9926a59b9bcd041b, scope.project.id=22414db73cef49549c4930bec9f6b5dc, user.id=a607948d7f414d0caf2eaee9eb0ec330) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:38.430555 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08336178-c0b4-47f9-9798-f441a7ac9ca7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:38.432013 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08336178-c0b4-47f9-9798-f441a7ac9ca7 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:38.442566 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 151/294] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:34:38 2022] GET /identity/v3/role_assignments?user.id=a607948d7f414d0caf2eaee9eb0ec330&scope.project.id=22414db73cef49549c4930bec9f6b5dc&role.id=5032636d8729461c9926a59b9bcd041b => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:38.555859 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c420105-6841-478b-9660-9580376d3513 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:38.556180 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c420105-6841-478b-9660-9580376d3513 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:38.556358 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c420105-6841-478b-9660-9580376d3513 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:38.558057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 144/295] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:38.564544 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab572496-f2de-4cbc-9db1-8760a5e92a7d None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:38.564849 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab572496-f2de-4cbc-9db1-8760a5e92a7d None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:38.565105 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab572496-f2de-4cbc-9db1-8760a5e92a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:38.589867 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-ab572496-f2de-4cbc-9db1-8760a5e92a7d None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:38.591208 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-ab572496-f2de-4cbc-9db1-8760a5e92a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:38.663718 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:38.669975 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 152/296] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:38.709018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2699d0-dfbc-4808-a21b-c625b2396346 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:38.709349 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2699d0-dfbc-4808-a21b-c625b2396346 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:38.709560 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2699d0-dfbc-4808-a21b-c625b2396346 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:38.710927 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 145/297] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:38.716379 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33d2c7-2ffc-4e19-8370-a9817d70f55a None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:38.716839 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33d2c7-2ffc-4e19-8370-a9817d70f55a None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:38.717264 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33d2c7-2ffc-4e19-8370-a9817d70f55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:38.741416 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-6f33d2c7-2ffc-4e19-8370-a9817d70f55a None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:38.742649 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-6f33d2c7-2ffc-4e19-8370-a9817d70f55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:38.812615 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 153/298] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:34:38.896127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b635210-ceec-4091-b4ab-f4fb3f5dfb30 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:38.896486 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b635210-ceec-4091-b4ab-f4fb3f5dfb30 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:38.896699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b635210-ceec-4091-b4ab-f4fb3f5dfb30 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:38.898303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 146/299] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:38.903542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250f0d6-7ea6-49e5-ba4c-8163129f2984 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:38.903834 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250f0d6-7ea6-49e5-ba4c-8163129f2984 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:38.904113 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250f0d6-7ea6-49e5-ba4c-8163129f2984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:38.924093 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-f250f0d6-7ea6-49e5-ba4c-8163129f2984 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:38.925354 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f250f0d6-7ea6-49e5-ba4c-8163129f2984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:38.991612 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b635210-ceec-4091-b4ab-f4fb3f5dfb30 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:38.997215 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 154/300] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:39.027052 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:39.108934 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b635210-ceec-4091-b4ab-f4fb3f5dfb30 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:39.138864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250f0d6-7ea6-49e5-ba4c-8163129f2984 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:39.148738 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae4165-62d8-497a-99bd-39550d58aff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:39.150169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae4165-62d8-497a-99bd-39550d58aff4 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:39.150638 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae4165-62d8-497a-99bd-39550d58aff4 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:39.151095 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae4165-62d8-497a-99bd-39550d58aff4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:39.165667 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae4165-62d8-497a-99bd-39550d58aff4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:39.172847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-88ae4165-62d8-497a-99bd-39550d58aff4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 00:34:39.172847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:39.172847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:39.172847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:39.172847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:39.172847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:39.172847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:39.172847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:39.172847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:39.172847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:39.172847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:39.172847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:39.172847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:39.172847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:39.172847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:39.172847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:39.173832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:39.173832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:39.173832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:39.173832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:39.173832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:39.173832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:39.173832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:39.173832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:39.173832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:39.173832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:39.173832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:39.173832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:39.173832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:39.173832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:39.173832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:39.173832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:39.174669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:39.174669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:39.174669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:39.174669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:39.174669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:39.174669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:39.174669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:39.174669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:39.174669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:39.174669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:39.174669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 00:34:39.174669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:34:39.174669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 147/301] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:39.181918 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae4165-62d8-497a-99bd-39550d58aff4 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:39.218644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:39.271061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250f0d6-7ea6-49e5-ba4c-8163129f2984 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:39.277059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:39.294985 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae4165-62d8-497a-99bd-39550d58aff4 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:39.310937 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152a1a3-5d0e-46f4-907e-78d8f1297ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:39.311587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152a1a3-5d0e-46f4-907e-78d8f1297ccf None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:39.311869 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152a1a3-5d0e-46f4-907e-78d8f1297ccf None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:39.312153 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152a1a3-5d0e-46f4-907e-78d8f1297ccf None admin] PATH_INFO: `/v3/domains/default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:39.313266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e152a1a3-5d0e-46f4-907e-78d8f1297ccf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:39.315298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e152a1a3-5d0e-46f4-907e-78d8f1297ccf None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:39.321477 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 155/302] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:34:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:39.321477 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152a1a3-5d0e-46f4-907e-78d8f1297ccf None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:39.338876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe1cebf-7bd1-4596-a6c0-6d830b994355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:39.339483 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1cebf-7bd1-4596-a6c0-6d830b994355 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:39.339818 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1cebf-7bd1-4596-a6c0-6d830b994355 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:39.340095 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1cebf-7bd1-4596-a6c0-6d830b994355 None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:39.340797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe1cebf-7bd1-4596-a6c0-6d830b994355 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:39.341115 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe1cebf-7bd1-4596-a6c0-6d830b994355 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:39.342473 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe1cebf-7bd1-4596-a6c0-6d830b994355 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:39.351524 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 148/303] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:34:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:39.352295 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe1cebf-7bd1-4596-a6c0-6d830b994355 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:39.431475 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152a1a3-5d0e-46f4-907e-78d8f1297ccf None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:39.460778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:39.463094 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe1cebf-7bd1-4596-a6c0-6d830b994355 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:39.469194 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7a1c0-c0cc-4e7a-b767-4f976d701409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:39.469866 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7a1c0-c0cc-4e7a-b767-4f976d701409 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:39.470235 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7a1c0-c0cc-4e7a-b767-4f976d701409 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:39.470512 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7a1c0-c0cc-4e7a-b767-4f976d701409 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:39.480876 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e7a1c0-c0cc-4e7a-b767-4f976d701409 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:39.486149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-c5e7a1c0-c0cc-4e7a-b767-4f976d701409 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:39.486149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:39.486149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:39.486149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:39.486149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:39.486149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:39.486149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:39.486149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:39.486149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:39.486149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:39.486149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:39.486149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:39.486149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:39.486149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:39.486149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:39.486149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:39.487113 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:39.487113 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:39.487113 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:39.487113 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:39.487113 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:39.487113 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:39.487113 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:39.487113 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:39.487113 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:39.487113 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:39.487113 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:39.487113 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:39.487113 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:39.487113 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:39.487113 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:39.487113 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:39.489384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:39.489384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:39.489384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:39.489384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:39.489384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:39.489384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:39.489384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:39.489384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:39.489384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:39.489384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:39.489384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:39.489384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:34:39.489384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 156/304] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:39.489384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7a1c0-c0cc-4e7a-b767-4f976d701409 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:39.506256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f8d729-0364-4675-b5b6-56c3c95570da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:39.506876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f8d729-0364-4675-b5b6-56c3c95570da None admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:39.507154 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f8d729-0364-4675-b5b6-56c3c95570da None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:39.507429 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f8d729-0364-4675-b5b6-56c3c95570da None admin] PATH_INFO: `/v3/groups` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:39.508049 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f8d729-0364-4675-b5b6-56c3c95570da None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:39.509522 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f8d729-0364-4675-b5b6-56c3c95570da None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:39.589767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7a1c0-c0cc-4e7a-b767-4f976d701409 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:39.608065 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 149/305] 10.222.0.59 () {60 vars in 1134 bytes} [Fri Aug 26 00:34:39 2022] POST /identity/v3/groups => generated 229 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:34:39.608065 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f8d729-0364-4675-b5b6-56c3c95570da None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:39.609214 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:39.644784 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ad401-e88d-4a89-b7cd-f4bf9c0aca5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:39.645438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ad401-e88d-4a89-b7cd-f4bf9c0aca5f None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:39.645718 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ad401-e88d-4a89-b7cd-f4bf9c0aca5f None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:39.645992 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ad401-e88d-4a89-b7cd-f4bf9c0aca5f None admin] PATH_INFO: `/v3/users/9de7061e234b477584f0128f48aeaa95` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:39.662564 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ad401-e88d-4a89-b7cd-f4bf9c0aca5f None admin] RBAC: Authorizing `identity:get_user(user_id=9de7061e234b477584f0128f48aeaa95)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:39.664167 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ad401-e88d-4a89-b7cd-f4bf9c0aca5f None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:39.666284 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 157/306] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:39 2022] GET /identity/v3/users/9de7061e234b477584f0128f48aeaa95 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:39.673701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ad401-e88d-4a89-b7cd-f4bf9c0aca5f None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:39.717105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f8d729-0364-4675-b5b6-56c3c95570da None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:39.727688 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:39.755370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd98cf1-3cfd-48c7-b109-74d9b65e4d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:39.755989 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd98cf1-3cfd-48c7-b109-74d9b65e4d54 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:39.756266 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd98cf1-3cfd-48c7-b109-74d9b65e4d54 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:39.756569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd98cf1-3cfd-48c7-b109-74d9b65e4d54 None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:39.760254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd98cf1-3cfd-48c7-b109-74d9b65e4d54 None admin] RBAC: Adding query filter params (name=member) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:39.760254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd98cf1-3cfd-48c7-b109-74d9b65e4d54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:39.760254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd98cf1-3cfd-48c7-b109-74d9b65e4d54 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:39.773030 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 150/307] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:34:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:39.775181 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd98cf1-3cfd-48c7-b109-74d9b65e4d54 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:39.794729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ad401-e88d-4a89-b7cd-f4bf9c0aca5f None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:39.839817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988befb-909f-45d0-b8cd-8ccc3967e329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:39.840432 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988befb-909f-45d0-b8cd-8ccc3967e329 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:39.840742 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988befb-909f-45d0-b8cd-8ccc3967e329 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:39.841020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988befb-909f-45d0-b8cd-8ccc3967e329 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:39.841759 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d988befb-909f-45d0-b8cd-8ccc3967e329 None admin] RBAC: Adding query filter params (role.id=b68669d0f20b4ac8a1a7d46142b03424, user.id=9de7061e234b477584f0128f48aeaa95, scope.system=all) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:39.842074 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d988befb-909f-45d0-b8cd-8ccc3967e329 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:39.844318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d988befb-909f-45d0-b8cd-8ccc3967e329 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:39.852578 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 158/308] 10.222.0.59 () {56 vars in 1308 bytes} [Fri Aug 26 00:34:39 2022] GET /identity/v3/role_assignments?user.id=9de7061e234b477584f0128f48aeaa95&scope.system=all&role.id=b68669d0f20b4ac8a1a7d46142b03424 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:39.868903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:39.900192 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd98cf1-3cfd-48c7-b109-74d9b65e4d54 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:39.938133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538de3f-fa09-4572-b730-bc2b9dd4b388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:39.939708 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538de3f-fa09-4572-b730-bc2b9dd4b388 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:39.940185 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538de3f-fa09-4572-b730-bc2b9dd4b388 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:39.940690 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538de3f-fa09-4572-b730-bc2b9dd4b388 None admin] PATH_INFO: `/v3/users/8bae51e1801d4f70b24f2fdf30c93ddd` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:39.942153 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538de3f-fa09-4572-b730-bc2b9dd4b388 None admin] RBAC: Authorizing `identity:get_user(user_id=8bae51e1801d4f70b24f2fdf30c93ddd)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:39.943879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538de3f-fa09-4572-b730-bc2b9dd4b388 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:39.945820 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 151/309] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:39 2022] GET /identity/v3/users/8bae51e1801d4f70b24f2fdf30c93ddd => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:39.953652 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988befb-909f-45d0-b8cd-8ccc3967e329 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:40.071776 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988befb-909f-45d0-b8cd-8ccc3967e329 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:40.082302 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:40.107598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c28f0-f28e-46d4-afc6-b9a753fc4d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:40.108374 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c28f0-f28e-46d4-afc6-b9a753fc4d4a None admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:40.108741 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c28f0-f28e-46d4-afc6-b9a753fc4d4a None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:40.109077 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c28f0-f28e-46d4-afc6-b9a753fc4d4a None admin] PATH_INFO: `/v3/system/users/8bae51e1801d4f70b24f2fdf30c93ddd/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:40.110541 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c28f0-f28e-46d4-afc6-b9a753fc4d4a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8bae51e1801d4f70b24f2fdf30c93ddd, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:40.111916 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c28f0-f28e-46d4-afc6-b9a753fc4d4a None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:40.184091 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb8831-0756-4999-95dc-55bd9ebfdd5b None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:40.184407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb8831-0756-4999-95dc-55bd9ebfdd5b None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:40.184681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb8831-0756-4999-95dc-55bd9ebfdd5b None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:40.186197 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 152/310] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:40.191220 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db50608-94a3-4f1b-86f7-1ac1aa55815c None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:40.191512 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db50608-94a3-4f1b-86f7-1ac1aa55815c None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:40.191800 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db50608-94a3-4f1b-86f7-1ac1aa55815c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:40.210187 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 159/311] 10.222.0.59 () {58 vars in 1334 bytes} [Fri Aug 26 00:34:39 2022] PUT /identity/v3/system/users/8bae51e1801d4f70b24f2fdf30c93ddd/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:34:40.210695 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-7db50608-94a3-4f1b-86f7-1ac1aa55815c None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:40.216776 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-7db50608-94a3-4f1b-86f7-1ac1aa55815c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:40.298005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 153/312] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:34:40.572686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c28f0-f28e-46d4-afc6-b9a753fc4d4a None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:40.677889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c28f0-f28e-46d4-afc6-b9a753fc4d4a None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:40.713451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe372da-00d7-4682-bba3-fc2fd7e57ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:40.714014 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe372da-00d7-4682-bba3-fc2fd7e57ad6 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:40.714292 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe372da-00d7-4682-bba3-fc2fd7e57ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:40.714564 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe372da-00d7-4682-bba3-fc2fd7e57ad6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:40.725221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe372da-00d7-4682-bba3-fc2fd7e57ad6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:40.730390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-afe372da-00d7-4682-bba3-fc2fd7e57ad6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 00:34:40.730390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:40.730390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:40.730390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:40.730390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:40.730390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:40.730390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:40.730390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:40.730390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:40.730390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:40.730390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:40.730390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:40.730390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:40.730390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:40.730390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:40.730390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:40.731260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:40.731260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:40.731260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:40.731260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:40.731260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:40.731260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:40.731260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:40.731260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:40.731260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:40.731260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:40.731260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:40.731260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:40.731260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:40.731260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:40.731260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:40.731260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:40.733520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:40.733520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:40.733520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:40.733520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:40.733520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:40.733520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:40.733520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:40.733520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:40.733520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:40.733520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:40.733520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 00:34:40.733520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:34:40.733520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 160/313] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:40.741084 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db50608-94a3-4f1b-86f7-1ac1aa55815c None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:40.818459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:40.869455 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db50608-94a3-4f1b-86f7-1ac1aa55815c None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:40.912265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1927716-cb97-4b70-ba9d-5c9ec60c3d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:40.913252 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1927716-cb97-4b70-ba9d-5c9ec60c3d4b None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:40.913740 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1927716-cb97-4b70-ba9d-5c9ec60c3d4b None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:40.914237 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1927716-cb97-4b70-ba9d-5c9ec60c3d4b None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:40.915104 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1927716-cb97-4b70-ba9d-5c9ec60c3d4b None admin] RBAC: Adding query filter params (name=reader) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:40.915607 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1927716-cb97-4b70-ba9d-5c9ec60c3d4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:40.917346 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1927716-cb97-4b70-ba9d-5c9ec60c3d4b None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:40.926674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 154/314] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:34:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:40.932670 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe372da-00d7-4682-bba3-fc2fd7e57ad6 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:40.977638 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:41.052023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe372da-00d7-4682-bba3-fc2fd7e57ad6 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:41.100456 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d2203b-1111-474a-8fb2-5288cff1369a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:41.101135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2203b-1111-474a-8fb2-5288cff1369a None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:41.101412 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2203b-1111-474a-8fb2-5288cff1369a None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:41.101686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2203b-1111-474a-8fb2-5288cff1369a None admin] PATH_INFO: `/v3/users/548e7a814507494bb420b223caffaaab` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:41.102810 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d2203b-1111-474a-8fb2-5288cff1369a None admin] RBAC: Authorizing `identity:get_user(user_id=548e7a814507494bb420b223caffaaab)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:41.104261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d2203b-1111-474a-8fb2-5288cff1369a None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:41.105854 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 161/315] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:40 2022] GET /identity/v3/users/548e7a814507494bb420b223caffaaab => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:41.111872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1927716-cb97-4b70-ba9d-5c9ec60c3d4b None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:41.235639 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1927716-cb97-4b70-ba9d-5c9ec60c3d4b None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:41.285737 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:41.305767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82457ea-558b-405b-9173-e22e6ae3beb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:41.306382 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82457ea-558b-405b-9173-e22e6ae3beb0 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:41.307485 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82457ea-558b-405b-9173-e22e6ae3beb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:41.307485 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82457ea-558b-405b-9173-e22e6ae3beb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:41.446667 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82457ea-558b-405b-9173-e22e6ae3beb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:41.448135 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82457ea-558b-405b-9173-e22e6ae3beb0 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:41.551330 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 155/316] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:34:41.557339 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d2203b-1111-474a-8fb2-5288cff1369a None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:41.665503 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d2203b-1111-474a-8fb2-5288cff1369a None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:41.696974 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:41.706964 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e3e59-91b5-4665-96c2-5e6c28171039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:41.707910 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e3e59-91b5-4665-96c2-5e6c28171039 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:41.708448 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e3e59-91b5-4665-96c2-5e6c28171039 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:41.721049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e3e59-91b5-4665-96c2-5e6c28171039 None admin] PATH_INFO: `/v3/projects/22414db73cef49549c4930bec9f6b5dc` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:41.721049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e3e59-91b5-4665-96c2-5e6c28171039 None admin] RBAC: Authorizing `identity:get_project(project_id=22414db73cef49549c4930bec9f6b5dc)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:41.721049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e3e59-91b5-4665-96c2-5e6c28171039 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:41.721049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 162/317] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:41 2022] GET /identity/v3/projects/22414db73cef49549c4930bec9f6b5dc => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:41.721328 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82457ea-558b-405b-9173-e22e6ae3beb0 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:41.748669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:41.754102 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808ca3d-55b4-4e13-99e6-35bc34581658 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:41.754391 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808ca3d-55b4-4e13-99e6-35bc34581658 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:41.754765 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808ca3d-55b4-4e13-99e6-35bc34581658 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:41.756233 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 163/318] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:34:41.762148 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb365b-7793-424f-962e-fbe3f1a8b056 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:41.762635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb365b-7793-424f-962e-fbe3f1a8b056 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:41.763071 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb365b-7793-424f-962e-fbe3f1a8b056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:41.787363 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-72eb365b-7793-424f-962e-fbe3f1a8b056 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:41.788758 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-72eb365b-7793-424f-962e-fbe3f1a8b056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:41.825420 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82457ea-558b-405b-9173-e22e6ae3beb0 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:41.861705 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6ecb0-c134-448b-884c-ee29a827d34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:41.862374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6ecb0-c134-448b-884c-ee29a827d34a None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:41.862591 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6ecb0-c134-448b-884c-ee29a827d34a None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:41.862978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6ecb0-c134-448b-884c-ee29a827d34a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:41.863197 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 164/319] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:34:41.865233 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f6ecb0-c134-448b-884c-ee29a827d34a None admin] RBAC: Adding query filter params (role.id=b68669d0f20b4ac8a1a7d46142b03424, scope.project.id=22414db73cef49549c4930bec9f6b5dc, user.id=548e7a814507494bb420b223caffaaab) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:41.865752 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f6ecb0-c134-448b-884c-ee29a827d34a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:41.867375 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f6ecb0-c134-448b-884c-ee29a827d34a None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:41.874291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 156/320] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:34:41 2022] GET /identity/v3/role_assignments?user.id=548e7a814507494bb420b223caffaaab&scope.project.id=22414db73cef49549c4930bec9f6b5dc&role.id=b68669d0f20b4ac8a1a7d46142b03424 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:41.942032 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20de03-d68c-49a4-9e07-4fafc0352f5c None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:41.942683 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20de03-d68c-49a4-9e07-4fafc0352f5c None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:41.943161 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20de03-d68c-49a4-9e07-4fafc0352f5c None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:41.945101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 165/321] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:41.950397 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba390a-2d0e-4378-bf34-9f67ecddd974 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:41.950681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba390a-2d0e-4378-bf34-9f67ecddd974 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:41.950942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba390a-2d0e-4378-bf34-9f67ecddd974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:41.970575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-87ba390a-2d0e-4378-bf34-9f67ecddd974 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:41.971772 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-87ba390a-2d0e-4378-bf34-9f67ecddd974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:42.046201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 157/322] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:42.143656 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20de03-d68c-49a4-9e07-4fafc0352f5c None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:42.243222 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:42.248697 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20de03-d68c-49a4-9e07-4fafc0352f5c None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:42.283644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740679df-d487-4668-b8ff-7ba5ad4226d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:42.284282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740679df-d487-4668-b8ff-7ba5ad4226d7 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:42.284654 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740679df-d487-4668-b8ff-7ba5ad4226d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:42.284914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740679df-d487-4668-b8ff-7ba5ad4226d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:42.286033 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740679df-d487-4668-b8ff-7ba5ad4226d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:42.287501 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740679df-d487-4668-b8ff-7ba5ad4226d7 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:42.289275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 166/323] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:34:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:42.294805 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba390a-2d0e-4378-bf34-9f67ecddd974 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:42.311210 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740679df-d487-4668-b8ff-7ba5ad4226d7 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:42.321859 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:42.396571 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba390a-2d0e-4378-bf34-9f67ecddd974 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:42.411004 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740679df-d487-4668-b8ff-7ba5ad4226d7 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:42.431061 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a632e6f4-5e53-43a4-bc9c-f57ff35d6187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:42.431674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632e6f4-5e53-43a4-bc9c-f57ff35d6187 None admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:42.431960 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632e6f4-5e53-43a4-bc9c-f57ff35d6187 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:42.432379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632e6f4-5e53-43a4-bc9c-f57ff35d6187 None admin] PATH_INFO: `/v3/groups` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:42.433035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a632e6f4-5e53-43a4-bc9c-f57ff35d6187 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:42.434488 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a632e6f4-5e53-43a4-bc9c-f57ff35d6187 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:42.445750 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e21c0-1bb4-4b32-98f4-42ec3551e911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:42.446380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e21c0-1bb4-4b32-98f4-42ec3551e911 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:42.446657 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e21c0-1bb4-4b32-98f4-42ec3551e911 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:42.446925 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e21c0-1bb4-4b32-98f4-42ec3551e911 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:42.456766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e21c0-1bb4-4b32-98f4-42ec3551e911 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:42.461495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-e04e21c0-1bb4-4b32-98f4-42ec3551e911 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 00:34:42.461495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:42.461495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:42.461495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:42.461495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:42.461495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:42.461495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:42.461495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:42.461495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:42.461495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:42.461495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:42.461495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:42.461495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:42.461495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:42.461495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:42.461495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:42.462466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:42.462466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:42.462466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:42.462466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:42.462466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:42.462466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:42.462466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:42.462466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:42.462466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:42.462466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:42.462466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:42.462466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:42.462466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:42.462466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:42.462466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:42.462466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:42.463507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:42.463507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:42.463507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:42.463507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:42.463507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:42.463507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:42.463507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:42.463507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:42.463507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:42.463507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:42.463507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 00:34:42.463507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:34:42.463507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 167/324] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:42.465726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10a694-f746-4081-a50d-51a881686170 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:42.466014 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10a694-f746-4081-a50d-51a881686170 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:42.466288 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10a694-f746-4081-a50d-51a881686170 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:42.467587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 168/325] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:42.468295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10a694-f746-4081-a50d-51a881686170 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:42.500285 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:42.511763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 158/326] 10.222.0.59 () {60 vars in 1134 bytes} [Fri Aug 26 00:34:42 2022] POST /identity/v3/groups => generated 226 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:34:42.513072 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da347ca-1d43-4c88-82e8-6cbca506e400 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:42.513460 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da347ca-1d43-4c88-82e8-6cbca506e400 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:42.513740 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da347ca-1d43-4c88-82e8-6cbca506e400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:42.541757 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-7da347ca-1d43-4c88-82e8-6cbca506e400 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:42.543132 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-7da347ca-1d43-4c88-82e8-6cbca506e400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:42.597804 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10a694-f746-4081-a50d-51a881686170 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:42.622665 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 159/327] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:42.635211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0cd0f-d491-4466-9020-dd3fa71e16c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:42.635950 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0cd0f-d491-4466-9020-dd3fa71e16c4 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:42.636240 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0cd0f-d491-4466-9020-dd3fa71e16c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:42.636542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0cd0f-d491-4466-9020-dd3fa71e16c4 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:42.637228 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c0cd0f-d491-4466-9020-dd3fa71e16c4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:42.637533 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c0cd0f-d491-4466-9020-dd3fa71e16c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:42.638940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c0cd0f-d491-4466-9020-dd3fa71e16c4 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:42.652602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 169/328] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:34:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:42.653955 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da347ca-1d43-4c88-82e8-6cbca506e400 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:42.671329 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:42.767929 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da347ca-1d43-4c88-82e8-6cbca506e400 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:42.804602 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c129a55-ad0a-4762-8de2-b5bbe63d065f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:42.805307 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c129a55-ad0a-4762-8de2-b5bbe63d065f None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:42.805595 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c129a55-ad0a-4762-8de2-b5bbe63d065f None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:42.805872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c129a55-ad0a-4762-8de2-b5bbe63d065f None admin] PATH_INFO: `/v3/users/9de7061e234b477584f0128f48aeaa95` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:42.807004 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c129a55-ad0a-4762-8de2-b5bbe63d065f None admin] RBAC: Authorizing `identity:get_user(user_id=9de7061e234b477584f0128f48aeaa95)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:42.808448 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c129a55-ad0a-4762-8de2-b5bbe63d065f None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:42.810497 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 160/329] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:42 2022] GET /identity/v3/users/9de7061e234b477584f0128f48aeaa95 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:42.817423 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0cd0f-d491-4466-9020-dd3fa71e16c4 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:42.893296 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c129a55-ad0a-4762-8de2-b5bbe63d065f None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:42.945008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:42.961901 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0cd0f-d491-4466-9020-dd3fa71e16c4 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:43.005631 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33379103-ef87-4801-93b1-e01cc15c4b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:43.006436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33379103-ef87-4801-93b1-e01cc15c4b87 None admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:43.006844 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33379103-ef87-4801-93b1-e01cc15c4b87 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:43.007132 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33379103-ef87-4801-93b1-e01cc15c4b87 None admin] PATH_INFO: `/v3/system/users/9de7061e234b477584f0128f48aeaa95/roles/b68669d0f20b4ac8a1a7d46142b03424` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:43.008895 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:43.009082 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33379103-ef87-4801-93b1-e01cc15c4b87 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9de7061e234b477584f0128f48aeaa95, role_id=b68669d0f20b4ac8a1a7d46142b03424)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:43.011571 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33379103-ef87-4801-93b1-e01cc15c4b87 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:43.031185 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c129a55-ad0a-4762-8de2-b5bbe63d065f None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:43.068198 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14036116-175e-4e4e-b136-d8c830a5bffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:43.068967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14036116-175e-4e4e-b136-d8c830a5bffd None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:43.069243 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14036116-175e-4e4e-b136-d8c830a5bffd None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:43.069566 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14036116-175e-4e4e-b136-d8c830a5bffd None admin] PATH_INFO: `/v3/roles/member` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:43.083944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14036116-175e-4e4e-b136-d8c830a5bffd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:43.088760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-14036116-175e-4e4e-b136-d8c830a5bffd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:43.088760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:43.088760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:43.088760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:43.088760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:43.088760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:43.088760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:43.088760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:43.088760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:43.088760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:43.088760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:43.088760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:43.088760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:43.088760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:43.088760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:43.088760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:43.089721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:43.089721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:43.089721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:43.089721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:43.089721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:43.089721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:43.089721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:43.089721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:43.089721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:43.089721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:43.089721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:43.089721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:43.089721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:43.089721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:43.089721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:43.089721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:43.090538 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:43.090538 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:43.090538 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:43.090538 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:43.090538 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:43.090538 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:43.090538 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:43.090538 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:43.090538 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:43.090538 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:43.090538 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:43.090538 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:34:43.090538 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 161/330] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:34:43.095930 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14036116-175e-4e4e-b136-d8c830a5bffd None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:43.096901 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 170/331] 10.222.0.59 () {58 vars in 1334 bytes} [Fri Aug 26 00:34:42 2022] PUT /identity/v3/system/users/9de7061e234b477584f0128f48aeaa95/roles/b68669d0f20b4ac8a1a7d46142b03424 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:34:43.245818 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14036116-175e-4e4e-b136-d8c830a5bffd None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:43.286199 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df7b57-8088-4857-838f-3bcccc577f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:43.287105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df7b57-8088-4857-838f-3bcccc577f44 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:43.287556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df7b57-8088-4857-838f-3bcccc577f44 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:43.288034 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df7b57-8088-4857-838f-3bcccc577f44 None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:43.288815 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df7b57-8088-4857-838f-3bcccc577f44 None admin] RBAC: Adding query filter params (name=member) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:43.289117 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df7b57-8088-4857-838f-3bcccc577f44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:43.290506 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df7b57-8088-4857-838f-3bcccc577f44 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:43.299566 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 162/332] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:34:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:43.304893 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33379103-ef87-4801-93b1-e01cc15c4b87 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:43.442755 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33379103-ef87-4801-93b1-e01cc15c4b87 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:43.487517 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa576fd-8f28-425e-b163-765fc6b7d3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:43.488419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa576fd-8f28-425e-b163-765fc6b7d3ed None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:43.488889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa576fd-8f28-425e-b163-765fc6b7d3ed None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:43.489306 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa576fd-8f28-425e-b163-765fc6b7d3ed None admin] PATH_INFO: `/v3/users/8bae51e1801d4f70b24f2fdf30c93ddd` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:43.490702 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa576fd-8f28-425e-b163-765fc6b7d3ed None admin] RBAC: Authorizing `identity:get_user(user_id=8bae51e1801d4f70b24f2fdf30c93ddd)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:43.492298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa576fd-8f28-425e-b163-765fc6b7d3ed None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:43.494054 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 171/333] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:43 2022] GET /identity/v3/users/8bae51e1801d4f70b24f2fdf30c93ddd => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:43.500306 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df7b57-8088-4857-838f-3bcccc577f44 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:43.503692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:43.634963 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df7b57-8088-4857-838f-3bcccc577f44 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:43.682546 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0128f10e-befd-422a-bbf0-716fec9e2a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:43.683157 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128f10e-befd-422a-bbf0-716fec9e2a0e None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:43.683433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128f10e-befd-422a-bbf0-716fec9e2a0e None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:43.683702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128f10e-befd-422a-bbf0-716fec9e2a0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:43.684420 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0128f10e-befd-422a-bbf0-716fec9e2a0e None admin] RBAC: Adding query filter params (role.id=5032636d8729461c9926a59b9bcd041b, user.id=8bae51e1801d4f70b24f2fdf30c93ddd, scope.system=all) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:43.684746 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0128f10e-befd-422a-bbf0-716fec9e2a0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:43.686161 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0128f10e-befd-422a-bbf0-716fec9e2a0e None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:43.704267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 163/334] 10.222.0.59 () {56 vars in 1308 bytes} [Fri Aug 26 00:34:43 2022] GET /identity/v3/role_assignments?user.id=8bae51e1801d4f70b24f2fdf30c93ddd&scope.system=all&role.id=5032636d8729461c9926a59b9bcd041b => generated 510 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:45.009108 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d96b69-e7a3-4888-9627-3b5321852740 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:45.010057 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d96b69-e7a3-4888-9627-3b5321852740 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:45.010496 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d96b69-e7a3-4888-9627-3b5321852740 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:45.012306 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 172/335] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:45.018274 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8fcd49-2192-4d2c-a854-3ae61dd16726 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:45.019167 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8fcd49-2192-4d2c-a854-3ae61dd16726 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:45.019695 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8fcd49-2192-4d2c-a854-3ae61dd16726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:45.043866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-2a8fcd49-2192-4d2c-a854-3ae61dd16726 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:45.045454 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-2a8fcd49-2192-4d2c-a854-3ae61dd16726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:45.120874 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:45.134800 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 164/336] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:45.297723 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaaf5e9-5e5b-4950-898a-971ff18e9338 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:45.298061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaaf5e9-5e5b-4950-898a-971ff18e9338 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:45.298238 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaaf5e9-5e5b-4950-898a-971ff18e9338 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:45.299794 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 173/337] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:45.305182 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10ca15-2d53-4dae-a9fa-5213ed3fdd72 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:45.305535 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10ca15-2d53-4dae-a9fa-5213ed3fdd72 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:45.305814 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10ca15-2d53-4dae-a9fa-5213ed3fdd72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:45.326445 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-3d10ca15-2d53-4dae-a9fa-5213ed3fdd72 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:45.327638 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-3d10ca15-2d53-4dae-a9fa-5213ed3fdd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:45.415195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 165/338] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:45.511547 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9ba97-b73b-4109-b5a8-4ff1a64131fb None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:45.512173 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9ba97-b73b-4109-b5a8-4ff1a64131fb None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:45.512708 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9ba97-b73b-4109-b5a8-4ff1a64131fb None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:45.514473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 174/339] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:45.530539 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f44e8a-028e-437a-a8e9-df0f4dafc806 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:45.530848 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f44e8a-028e-437a-a8e9-df0f4dafc806 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:45.531093 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f44e8a-028e-437a-a8e9-df0f4dafc806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:45.557617 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-b3f44e8a-028e-437a-a8e9-df0f4dafc806 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:45.558796 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-b3f44e8a-028e-437a-a8e9-df0f4dafc806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:45.574922 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9ba97-b73b-4109-b5a8-4ff1a64131fb None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:45.620511 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:45.650673 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 166/340] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:45.745552 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9ba97-b73b-4109-b5a8-4ff1a64131fb None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:45.785535 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e1c53-f1e5-46a8-984e-e48e19f9b59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:45.786562 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e1c53-f1e5-46a8-984e-e48e19f9b59c None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:45.787046 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e1c53-f1e5-46a8-984e-e48e19f9b59c None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:45.787520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e1c53-f1e5-46a8-984e-e48e19f9b59c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:45.799004 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e1c53-f1e5-46a8-984e-e48e19f9b59c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:45.804242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-d46e1c53-f1e5-46a8-984e-e48e19f9b59c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 00:34:45.804242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:45.804242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:45.804242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:45.804242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:45.804242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:45.804242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:45.804242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:45.804242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:45.804242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:45.804242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:45.804242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:45.804242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:45.804242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:45.804242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:45.804242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:45.805176 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:45.805176 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:45.805176 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:45.805176 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:45.805176 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:45.805176 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:45.805176 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:45.805176 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:45.805176 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:45.805176 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:45.805176 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:45.805176 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:45.805176 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:45.805176 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:45.805176 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:45.805176 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:45.806007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:45.806007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:45.806007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:45.806007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:45.806007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:45.806007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:45.806007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:45.806007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:45.806007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:45.806007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:45.806007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 00:34:45.806007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:34:45.808341 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 175/341] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Aug 26 00:34:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:45.817966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f44e8a-028e-437a-a8e9-df0f4dafc806 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:45.875781 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ac38e-5eff-4672-91f4-a909fd865aca None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:45.876086 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ac38e-5eff-4672-91f4-a909fd865aca None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:45.876350 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ac38e-5eff-4672-91f4-a909fd865aca None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:45.878094 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 176/342] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:34:45.884975 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ad58d-9ea1-4667-a99e-f255a8d1bc23 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:45.885260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ad58d-9ea1-4667-a99e-f255a8d1bc23 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:45.885852 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ad58d-9ea1-4667-a99e-f255a8d1bc23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:45.921460 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-cb7ad58d-9ea1-4667-a99e-f255a8d1bc23 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:45.923179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-cb7ad58d-9ea1-4667-a99e-f255a8d1bc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:45.941659 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:46.020737 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f44e8a-028e-437a-a8e9-df0f4dafc806 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:46.062630 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 177/343] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:46.064226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ad58d-9ea1-4667-a99e-f255a8d1bc23 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:46.065681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930afadc-6af7-40d1-88cd-071713bc4ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:46.066602 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930afadc-6af7-40d1-88cd-071713bc4ddd None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:46.066890 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930afadc-6af7-40d1-88cd-071713bc4ddd None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:46.067166 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930afadc-6af7-40d1-88cd-071713bc4ddd None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:46.067914 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930afadc-6af7-40d1-88cd-071713bc4ddd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:46.068222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930afadc-6af7-40d1-88cd-071713bc4ddd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:46.069763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930afadc-6af7-40d1-88cd-071713bc4ddd None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:46.087187 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 167/344] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:34:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:46.093377 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930afadc-6af7-40d1-88cd-071713bc4ddd None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:46.101486 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:46.236075 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930afadc-6af7-40d1-88cd-071713bc4ddd None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:46.251728 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:46.279502 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b17a8-c3a2-4ee0-9e59-212ae72627e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:46.280166 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b17a8-c3a2-4ee0-9e59-212ae72627e3 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:46.281038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b17a8-c3a2-4ee0-9e59-212ae72627e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:46.281321 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b17a8-c3a2-4ee0-9e59-212ae72627e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:46.303256 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ad58d-9ea1-4667-a99e-f255a8d1bc23 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:46.372402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47716ef7-04f4-4b80-8371-4a11224d30db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:46.373747 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47716ef7-04f4-4b80-8371-4a11224d30db None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:46.374024 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47716ef7-04f4-4b80-8371-4a11224d30db None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:46.374291 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47716ef7-04f4-4b80-8371-4a11224d30db None admin] PATH_INFO: `/v3/roles/member` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:46.393282 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b17a8-c3a2-4ee0-9e59-212ae72627e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:46.394757 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b17a8-c3a2-4ee0-9e59-212ae72627e3 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:46.404541 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47716ef7-04f4-4b80-8371-4a11224d30db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:46.412833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-47716ef7-04f4-4b80-8371-4a11224d30db None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:46.412833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:46.412833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:46.412833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:46.412833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:46.412833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:46.412833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:46.412833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:46.412833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:46.412833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:46.412833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:46.412833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:46.412833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:46.412833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:46.412833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:46.412833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:46.413766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:46.413766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:46.413766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:46.413766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:46.413766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:46.413766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:46.413766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:46.413766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:46.413766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:46.413766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:46.413766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:46.413766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:46.413766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:46.413766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:46.413766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:46.413766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:46.415406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:46.415406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:46.415406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:46.415406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:46.415406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:46.415406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:46.415406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:46.415406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:46.415406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:46.415406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:46.415406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:46.415406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:34:46.415406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 178/345] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:46.415406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47716ef7-04f4-4b80-8371-4a11224d30db None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:46.515690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:46.540122 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 168/346] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 00:34:46.541316 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b17a8-c3a2-4ee0-9e59-212ae72627e3 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:46.562001 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:46.567768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47716ef7-04f4-4b80-8371-4a11224d30db None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:46.610968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed5ef1-f97e-4038-a943-83e1423449df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:46.611862 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed5ef1-f97e-4038-a943-83e1423449df None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:46.612371 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed5ef1-f97e-4038-a943-83e1423449df None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:46.612820 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed5ef1-f97e-4038-a943-83e1423449df None admin] PATH_INFO: `/v3/roles/member` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:46.625161 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed5ef1-f97e-4038-a943-83e1423449df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:46.630318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-83ed5ef1-f97e-4038-a943-83e1423449df None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:46.630318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:46.630318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:46.630318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:46.630318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:46.630318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:46.630318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:46.630318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:46.630318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:46.630318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:46.630318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:46.630318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:46.630318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:46.630318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:46.630318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:46.630318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:46.631278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:46.631278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:46.631278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:46.631278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:46.631278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:46.631278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:46.631278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:46.631278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:46.631278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:46.631278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:46.631278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:46.631278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:46.631278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:46.631278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:46.631278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:46.631278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:46.632120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:46.632120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:46.632120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:46.632120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:46.632120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:46.632120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:46.632120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:46.632120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:46.632120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:46.632120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:46.632120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:46.632120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:34:46.636599 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 179/347] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:46.637500 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed5ef1-f97e-4038-a943-83e1423449df None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:46.678938 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b17a8-c3a2-4ee0-9e59-212ae72627e3 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:46.727775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40517249-68bd-4a8a-833e-9651e58628d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:46.728444 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40517249-68bd-4a8a-833e-9651e58628d2 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:46.728747 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40517249-68bd-4a8a-833e-9651e58628d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:46.729015 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40517249-68bd-4a8a-833e-9651e58628d2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:46.740830 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40517249-68bd-4a8a-833e-9651e58628d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:46.746145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-40517249-68bd-4a8a-833e-9651e58628d2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 00:34:46.746145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:46.746145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:46.746145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:46.746145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:46.746145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:46.746145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:46.746145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:46.746145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:46.746145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:46.746145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:46.746145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:46.746145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:46.746145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:46.746145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:46.746145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:46.747047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:46.747047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:46.747047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:46.747047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:46.747047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:46.747047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:46.747047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:46.747047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:46.747047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:46.747047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:46.747047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:46.747047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:46.747047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:46.747047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:46.747047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:46.747047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:46.747876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:46.747876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:46.747876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:46.747876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:46.747876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:46.747876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:46.747876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:46.747876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:46.747876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:46.747876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:46.747876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 00:34:46.747876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:34:46.747876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 169/348] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:46.748617 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40517249-68bd-4a8a-833e-9651e58628d2 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:46.768068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed5ef1-f97e-4038-a943-83e1423449df None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:46.808663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:46.816200 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9fe92-e883-413d-aac1-8cd22cedfc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:46.817028 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9fe92-e883-413d-aac1-8cd22cedfc67 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:46.817381 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9fe92-e883-413d-aac1-8cd22cedfc67 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:46.817727 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9fe92-e883-413d-aac1-8cd22cedfc67 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:46.818475 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c9fe92-e883-413d-aac1-8cd22cedfc67 None admin] RBAC: Adding query filter params (name=member) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:46.818850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c9fe92-e883-413d-aac1-8cd22cedfc67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:46.820503 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c9fe92-e883-413d-aac1-8cd22cedfc67 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:46.830497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 180/349] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:34:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:46.831243 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9fe92-e883-413d-aac1-8cd22cedfc67 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:46.838977 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:46.879248 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40517249-68bd-4a8a-833e-9651e58628d2 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:46.932070 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619aa179-7511-43b9-a89f-76e4720930cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:46.934932 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619aa179-7511-43b9-a89f-76e4720930cd None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:46.934932 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619aa179-7511-43b9-a89f-76e4720930cd None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:46.934932 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619aa179-7511-43b9-a89f-76e4720930cd None admin] PATH_INFO: `/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:46.936218 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619aa179-7511-43b9-a89f-76e4720930cd None admin] RBAC: Authorizing `identity:get_project(project_id=c60b6aa5591d4178ba9eb5fd05b7ee73)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:46.938619 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619aa179-7511-43b9-a89f-76e4720930cd None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:46.939800 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 170/350] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:46 2022] GET /identity/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:46.940528 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619aa179-7511-43b9-a89f-76e4720930cd None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:46.963739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9fe92-e883-413d-aac1-8cd22cedfc67 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:47.006102 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3d061-b0d6-4eb9-ae28-ed7addf9ffb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:47.006725 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3d061-b0d6-4eb9-ae28-ed7addf9ffb3 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:47.007012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3d061-b0d6-4eb9-ae28-ed7addf9ffb3 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:47.007282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3d061-b0d6-4eb9-ae28-ed7addf9ffb3 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:47.007953 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f3d061-b0d6-4eb9-ae28-ed7addf9ffb3 None admin] RBAC: Adding query filter params (name=member) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:47.011221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f3d061-b0d6-4eb9-ae28-ed7addf9ffb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:47.011221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f3d061-b0d6-4eb9-ae28-ed7addf9ffb3 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:47.019890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 181/351] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:34:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:47.020764 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3d061-b0d6-4eb9-ae28-ed7addf9ffb3 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:47.059942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619aa179-7511-43b9-a89f-76e4720930cd None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:47.089929 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:47.104931 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c95286-e71a-4e80-b62b-3ada53db19fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:47.105836 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c95286-e71a-4e80-b62b-3ada53db19fe None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:47.106298 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c95286-e71a-4e80-b62b-3ada53db19fe None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:47.106764 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c95286-e71a-4e80-b62b-3ada53db19fe None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:47.107438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c95286-e71a-4e80-b62b-3ada53db19fe None admin] RBAC: Adding query filter params (name=reader) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:47.107745 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c95286-e71a-4e80-b62b-3ada53db19fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:47.109214 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c95286-e71a-4e80-b62b-3ada53db19fe None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:47.122610 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 171/352] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:34:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:47.124685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c343b8b-939f-4be9-9ea1-e7f0da40a77c None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:47.125192 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c343b8b-939f-4be9-9ea1-e7f0da40a77c None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:47.125762 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c343b8b-939f-4be9-9ea1-e7f0da40a77c None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:47.127773 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 172/353] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:47.128970 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c343b8b-939f-4be9-9ea1-e7f0da40a77c None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:47.133896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:47.162732 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3d061-b0d6-4eb9-ae28-ed7addf9ffb3 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:47.211197 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5004c997-aa99-42e9-a600-32beee7bf252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:47.211878 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004c997-aa99-42e9-a600-32beee7bf252 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:47.212160 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004c997-aa99-42e9-a600-32beee7bf252 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:47.212458 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004c997-aa99-42e9-a600-32beee7bf252 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:47.267735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c343b8b-939f-4be9-9ea1-e7f0da40a77c None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:47.308392 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5004c997-aa99-42e9-a600-32beee7bf252 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:47.309907 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5004c997-aa99-42e9-a600-32beee7bf252 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:47.314394 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538ec21-46e6-41a4-8030-722139e64242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:47.315335 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538ec21-46e6-41a4-8030-722139e64242 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:47.315766 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538ec21-46e6-41a4-8030-722139e64242 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:47.316847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538ec21-46e6-41a4-8030-722139e64242 None admin] PATH_INFO: `/v3/groups/008b22bc962749f88dcab89d95b610bd` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:47.325458 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b538ec21-46e6-41a4-8030-722139e64242 None admin] RBAC: Authorizing `identity:get_group(group_id=008b22bc962749f88dcab89d95b610bd)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:47.327792 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b538ec21-46e6-41a4-8030-722139e64242 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:47.330761 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 173/354] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Aug 26 00:34:47 2022] GET /identity/v3/groups/008b22bc962749f88dcab89d95b610bd => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:47.331389 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538ec21-46e6-41a4-8030-722139e64242 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:47.379899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:47.427341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:47.452444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 182/355] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:34:47.453553 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5004c997-aa99-42e9-a600-32beee7bf252 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:47.486719 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538ec21-46e6-41a4-8030-722139e64242 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:47.529646 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de0afc-ac1c-4ce3-9aa3-2d6fe9bfebf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:47.530255 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de0afc-ac1c-4ce3-9aa3-2d6fe9bfebf4 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:47.530544 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de0afc-ac1c-4ce3-9aa3-2d6fe9bfebf4 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:47.530818 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de0afc-ac1c-4ce3-9aa3-2d6fe9bfebf4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:47.573782 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5004c997-aa99-42e9-a600-32beee7bf252 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:47.608973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de0afc-ac1c-4ce3-9aa3-2d6fe9bfebf4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:47.610545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de0afc-ac1c-4ce3-9aa3-2d6fe9bfebf4 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:47.619233 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe71961-b5cf-4e02-be46-9664e6e6c27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:47.619233 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe71961-b5cf-4e02-be46-9664e6e6c27b None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:47.619233 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe71961-b5cf-4e02-be46-9664e6e6c27b None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:47.619233 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe71961-b5cf-4e02-be46-9664e6e6c27b None admin] PATH_INFO: `/v3/users/9de7061e234b477584f0128f48aeaa95` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:47.619729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe71961-b5cf-4e02-be46-9664e6e6c27b None admin] RBAC: Authorizing `identity:get_user(user_id=9de7061e234b477584f0128f48aeaa95)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:47.621266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe71961-b5cf-4e02-be46-9664e6e6c27b None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:47.623244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 183/356] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:34:47 2022] GET /identity/v3/users/9de7061e234b477584f0128f48aeaa95 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:47.628588 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274104f5-a404-4562-a015-52ffdf78b6cf None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:47.628588 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274104f5-a404-4562-a015-52ffdf78b6cf None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:47.628588 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274104f5-a404-4562-a015-52ffdf78b6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:47.643164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-274104f5-a404-4562-a015-52ffdf78b6cf None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:47.645285 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-274104f5-a404-4562-a015-52ffdf78b6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:47.665596 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:47.694913 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:47.726661 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 184/357] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:47.727822 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274104f5-a404-4562-a015-52ffdf78b6cf None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:47.741522 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 174/358] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:34:47.744615 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de0afc-ac1c-4ce3-9aa3-2d6fe9bfebf4 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:47.869290 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274104f5-a404-4562-a015-52ffdf78b6cf None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:47.876643 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de0afc-ac1c-4ce3-9aa3-2d6fe9bfebf4 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:47.916319 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d7b3c-a728-41cd-8a8f-106c3715c4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:47.917026 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d7b3c-a728-41cd-8a8f-106c3715c4f4 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:47.917303 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d7b3c-a728-41cd-8a8f-106c3715c4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:47.917575 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d7b3c-a728-41cd-8a8f-106c3715c4f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:47.918299 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d7b3c-a728-41cd-8a8f-106c3715c4f4 None admin] RBAC: Adding query filter params (group.id=008b22bc962749f88dcab89d95b610bd, role.id=423aa6ac8b05402da7587e83e5c4ce09, scope.project.id=c60b6aa5591d4178ba9eb5fd05b7ee73) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:47.918604 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d7b3c-a728-41cd-8a8f-106c3715c4f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:47.919459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242cea53-69f8-4346-a3b4-b724901bbf5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:47.920022 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d7b3c-a728-41cd-8a8f-106c3715c4f4 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:47.920129 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242cea53-69f8-4346-a3b4-b724901bbf5e None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:47.920394 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242cea53-69f8-4346-a3b4-b724901bbf5e None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:47.920796 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242cea53-69f8-4346-a3b4-b724901bbf5e None admin] PATH_INFO: `/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:47.925253 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242cea53-69f8-4346-a3b4-b724901bbf5e None admin] RBAC: Authorizing `identity:get_project(project_id=c60b6aa5591d4178ba9eb5fd05b7ee73)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:47.926187 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242cea53-69f8-4346-a3b4-b724901bbf5e None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:47.926606 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 185/359] 10.222.0.59 () {56 vars in 1376 bytes} [Fri Aug 26 00:34:47 2022] GET /identity/v3/role_assignments?group.id=008b22bc962749f88dcab89d95b610bd&scope.project.id=c60b6aa5591d4178ba9eb5fd05b7ee73&role.id=423aa6ac8b05402da7587e83e5c4ce09 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:47.929145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fcd0a0-e100-48a1-8379-93ede7ede2c5 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:47.929145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fcd0a0-e100-48a1-8379-93ede7ede2c5 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:47.929145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fcd0a0-e100-48a1-8379-93ede7ede2c5 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:47.929494 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 186/360] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:47.933656 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fcd0a0-e100-48a1-8379-93ede7ede2c5 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:47.934883 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 175/361] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:47 2022] GET /identity/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:47.935742 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242cea53-69f8-4346-a3b4-b724901bbf5e None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:47.956680 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:47.993791 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:48.071046 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fcd0a0-e100-48a1-8379-93ede7ede2c5 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:48.076371 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242cea53-69f8-4346-a3b4-b724901bbf5e None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:48.119486 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb16fb-86dd-4b07-ae68-4e5beeeffcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:48.121486 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb16fb-86dd-4b07-ae68-4e5beeeffcfa None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:48.121965 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb16fb-86dd-4b07-ae68-4e5beeeffcfa None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:48.122533 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb16fb-86dd-4b07-ae68-4e5beeeffcfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:48.123268 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb16fb-86dd-4b07-ae68-4e5beeeffcfa None admin] RBAC: Adding query filter params (role.id=b68669d0f20b4ac8a1a7d46142b03424, user.id=9de7061e234b477584f0128f48aeaa95, scope.system=all) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:48.123575 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb16fb-86dd-4b07-ae68-4e5beeeffcfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:48.124594 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898e3e2-c4b7-4241-8e97-5d645c919272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:48.125041 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb16fb-86dd-4b07-ae68-4e5beeeffcfa None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:48.125234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898e3e2-c4b7-4241-8e97-5d645c919272 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:48.125516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898e3e2-c4b7-4241-8e97-5d645c919272 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:48.126573 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898e3e2-c4b7-4241-8e97-5d645c919272 None admin] PATH_INFO: `/v3/projects/22414db73cef49549c4930bec9f6b5dc` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:48.127497 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d898e3e2-c4b7-4241-8e97-5d645c919272 None admin] RBAC: Authorizing `identity:get_project(project_id=22414db73cef49549c4930bec9f6b5dc)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:48.129138 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d898e3e2-c4b7-4241-8e97-5d645c919272 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:48.130870 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 176/362] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:47 2022] GET /identity/v3/projects/22414db73cef49549c4930bec9f6b5dc => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:48.131906 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6baf1-ddb4-4608-b65e-ef14f8380399 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:48.132186 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6baf1-ddb4-4608-b65e-ef14f8380399 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:48.132531 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6baf1-ddb4-4608-b65e-ef14f8380399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:48.146473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 187/363] 10.222.0.59 () {56 vars in 1308 bytes} [Fri Aug 26 00:34:47 2022] GET /identity/v3/role_assignments?user.id=9de7061e234b477584f0128f48aeaa95&scope.system=all&role.id=b68669d0f20b4ac8a1a7d46142b03424 => generated 510 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:48.147333 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb16fb-86dd-4b07-ae68-4e5beeeffcfa None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:48.153795 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-79c6baf1-ddb4-4608-b65e-ef14f8380399 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:48.155017 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-79c6baf1-ddb4-4608-b65e-ef14f8380399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:48.259994 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 177/364] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:48.260908 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6baf1-ddb4-4608-b65e-ef14f8380399 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:48.269510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:48.322174 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb16fb-86dd-4b07-ae68-4e5beeeffcfa None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:48.328365 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:48.370380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1eb37-ec64-4fce-8210-362551f9b803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:48.371047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1eb37-ec64-4fce-8210-362551f9b803 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:48.371329 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1eb37-ec64-4fce-8210-362551f9b803 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:48.372081 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1eb37-ec64-4fce-8210-362551f9b803 None admin] PATH_INFO: `/v3/groups/008b22bc962749f88dcab89d95b610bd` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:48.373286 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1eb37-ec64-4fce-8210-362551f9b803 None admin] RBAC: Authorizing `identity:get_group(group_id=008b22bc962749f88dcab89d95b610bd)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:48.374744 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1eb37-ec64-4fce-8210-362551f9b803 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:48.376894 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 188/365] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Aug 26 00:34:48 2022] GET /identity/v3/groups/008b22bc962749f88dcab89d95b610bd => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:48.378052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1eb37-ec64-4fce-8210-362551f9b803 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:48.464806 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6baf1-ddb4-4608-b65e-ef14f8380399 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:48.514575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99379d95-5e3d-4179-b985-7a0bb7bbd21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:48.515168 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99379d95-5e3d-4179-b985-7a0bb7bbd21a None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:48.515502 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99379d95-5e3d-4179-b985-7a0bb7bbd21a None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:48.515786 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99379d95-5e3d-4179-b985-7a0bb7bbd21a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:48.519366 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1eb37-ec64-4fce-8210-362551f9b803 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:48.530777 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99379d95-5e3d-4179-b985-7a0bb7bbd21a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:48.538041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-99379d95-5e3d-4179-b985-7a0bb7bbd21a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 00:34:48.538041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:48.538041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:48.538041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:48.538041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:48.538041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:48.538041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:48.538041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:48.538041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:48.538041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:48.538041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:48.538041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:48.538041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:48.538041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:48.538041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:48.538041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:48.538973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:48.538973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:48.538973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:48.538973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:48.538973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:48.538973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:48.538973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:48.538973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:48.538973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:48.538973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:48.538973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:48.538973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:48.538973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:48.538973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:48.538973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:48.538973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:48.539790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:48.539790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:48.539790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:48.539790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:48.539790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:48.539790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:48.539790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:48.539790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:48.539790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:48.539790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:48.539790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 00:34:48.539790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:34:48.539790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 178/366] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Aug 26 00:34:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:48.540490 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99379d95-5e3d-4179-b985-7a0bb7bbd21a None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:48.557714 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d1674-c77d-43cd-94a8-4561415732a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:48.558357 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d1674-c77d-43cd-94a8-4561415732a5 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:48.558639 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d1674-c77d-43cd-94a8-4561415732a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:48.558970 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d1674-c77d-43cd-94a8-4561415732a5 None admin] PATH_INFO: `/v3/groups/008b22bc962749f88dcab89d95b610bd` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:48.560134 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d1674-c77d-43cd-94a8-4561415732a5 None admin] RBAC: Authorizing `identity:get_group(group_id=008b22bc962749f88dcab89d95b610bd)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:48.561618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d1674-c77d-43cd-94a8-4561415732a5 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:48.564022 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 189/367] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Aug 26 00:34:48 2022] GET /identity/v3/groups/008b22bc962749f88dcab89d95b610bd => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:48.564633 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d1674-c77d-43cd-94a8-4561415732a5 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:48.582381 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:48.654592 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:48.666364 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99379d95-5e3d-4179-b985-7a0bb7bbd21a None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:48.695839 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d1674-c77d-43cd-94a8-4561415732a5 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:48.709471 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a16bd7-aa20-4820-b206-ea46f5420022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:48.709996 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a16bd7-aa20-4820-b206-ea46f5420022 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:48.710437 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a16bd7-aa20-4820-b206-ea46f5420022 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:48.710722 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a16bd7-aa20-4820-b206-ea46f5420022 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:48.711456 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a16bd7-aa20-4820-b206-ea46f5420022 None admin] RBAC: Adding query filter params (group.id=008b22bc962749f88dcab89d95b610bd, role.id=5032636d8729461c9926a59b9bcd041b, scope.project.id=c60b6aa5591d4178ba9eb5fd05b7ee73) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:48.711769 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a16bd7-aa20-4820-b206-ea46f5420022 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:48.713806 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a16bd7-aa20-4820-b206-ea46f5420022 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:48.720027 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 179/368] 10.222.0.59 () {56 vars in 1376 bytes} [Fri Aug 26 00:34:48 2022] GET /identity/v3/role_assignments?group.id=008b22bc962749f88dcab89d95b610bd&scope.project.id=c60b6aa5591d4178ba9eb5fd05b7ee73&role.id=5032636d8729461c9926a59b9bcd041b => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:48.725510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a16bd7-aa20-4820-b206-ea46f5420022 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:48.737703 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0335347-0120-4abd-9f55-36c55405ba11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:48.738331 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0335347-0120-4abd-9f55-36c55405ba11 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:48.738614 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0335347-0120-4abd-9f55-36c55405ba11 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:48.738890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0335347-0120-4abd-9f55-36c55405ba11 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:48.739894 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0335347-0120-4abd-9f55-36c55405ba11 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:48.739894 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0335347-0120-4abd-9f55-36c55405ba11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:48.741884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0335347-0120-4abd-9f55-36c55405ba11 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:48.752228 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 190/369] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:34:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:48.753186 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0335347-0120-4abd-9f55-36c55405ba11 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:48.852200 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:48.861350 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a16bd7-aa20-4820-b206-ea46f5420022 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:48.895944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0335347-0120-4abd-9f55-36c55405ba11 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:48.909648 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfdb856-575e-43ab-b76b-21519cc3365c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:48.910271 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfdb856-575e-43ab-b76b-21519cc3365c None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:48.910818 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfdb856-575e-43ab-b76b-21519cc3365c None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:48.910927 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfdb856-575e-43ab-b76b-21519cc3365c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:48.911641 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfdb856-575e-43ab-b76b-21519cc3365c None admin] RBAC: Adding query filter params (group.id=008b22bc962749f88dcab89d95b610bd, role.id=5032636d8729461c9926a59b9bcd041b, scope.project.id=22414db73cef49549c4930bec9f6b5dc) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:48.912006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfdb856-575e-43ab-b76b-21519cc3365c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:48.913423 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfdb856-575e-43ab-b76b-21519cc3365c None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:48.920860 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 180/370] 10.222.0.59 () {56 vars in 1376 bytes} [Fri Aug 26 00:34:48 2022] GET /identity/v3/role_assignments?group.id=008b22bc962749f88dcab89d95b610bd&scope.project.id=22414db73cef49549c4930bec9f6b5dc&role.id=5032636d8729461c9926a59b9bcd041b => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:48.921761 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfdb856-575e-43ab-b76b-21519cc3365c None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:48.940682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca2eaf-7ea3-40dc-a39b-696a04dbf1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:48.941686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca2eaf-7ea3-40dc-a39b-696a04dbf1c5 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:48.942144 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca2eaf-7ea3-40dc-a39b-696a04dbf1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:48.942595 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca2eaf-7ea3-40dc-a39b-696a04dbf1c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:48.956932 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ca2eaf-7ea3-40dc-a39b-696a04dbf1c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:48.964679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-f1ca2eaf-7ea3-40dc-a39b-696a04dbf1c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:34:48.964679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:48.964679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:48.964679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:48.964679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:48.964679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:48.964679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:48.964679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:48.964679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:48.964679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:48.964679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:48.964679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:48.964679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:48.964679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:48.964679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:48.964679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:48.966582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:48.966582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:48.966582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:48.966582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:48.966582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:48.966582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:48.966582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:48.966582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:48.966582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:48.966582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:48.966582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:48.966582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:48.966582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:48.966582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:48.966582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:48.966582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:48.968190 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:48.968190 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:48.968190 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:48.968190 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:48.968190 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:48.968190 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:48.968190 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:48.968190 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:48.968190 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:48.968190 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:48.968190 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:34:48.968190 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:34:48.972564 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 191/371] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:34:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:48.974404 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:48.983074 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca2eaf-7ea3-40dc-a39b-696a04dbf1c5 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:49.108017 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfdb856-575e-43ab-b76b-21519cc3365c None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:49.158593 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf37e96-66e1-4a40-9cc6-c82c8ad499e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:49.159263 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf37e96-66e1-4a40-9cc6-c82c8ad499e0 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:49.159546 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf37e96-66e1-4a40-9cc6-c82c8ad499e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:49.159820 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf37e96-66e1-4a40-9cc6-c82c8ad499e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:49.179073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca2eaf-7ea3-40dc-a39b-696a04dbf1c5 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:49.229682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:49.233033 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1d78d-9532-43e8-861d-1d8f4f34f43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:49.234076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1d78d-9532-43e8-861d-1d8f4f34f43b None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:49.234361 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1d78d-9532-43e8-861d-1d8f4f34f43b None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:49.234685 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1d78d-9532-43e8-861d-1d8f4f34f43b None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:49.236307 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f1d78d-9532-43e8-861d-1d8f4f34f43b None admin] RBAC: Adding query filter params (name=admin) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:49.236740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f1d78d-9532-43e8-861d-1d8f4f34f43b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:49.238221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f1d78d-9532-43e8-861d-1d8f4f34f43b None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:49.250149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 192/372] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:49.258055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1d78d-9532-43e8-861d-1d8f4f34f43b None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:49.279232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf37e96-66e1-4a40-9cc6-c82c8ad499e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:49.280734 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf37e96-66e1-4a40-9cc6-c82c8ad499e0 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:49.307224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:49.408693 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 181/373] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:34:49.417051 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf37e96-66e1-4a40-9cc6-c82c8ad499e0 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:49.468238 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1d78d-9532-43e8-861d-1d8f4f34f43b None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:49.539393 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84029a9-26fc-4c43-a716-5215cf231094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:49.540021 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84029a9-26fc-4c43-a716-5215cf231094 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:49.540361 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84029a9-26fc-4c43-a716-5215cf231094 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:49.540662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84029a9-26fc-4c43-a716-5215cf231094 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:49.614748 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf37e96-66e1-4a40-9cc6-c82c8ad499e0 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:49.652077 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:49.661354 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:49.673616 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84029a9-26fc-4c43-a716-5215cf231094 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:49.675109 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84029a9-26fc-4c43-a716-5215cf231094 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:49.685363 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5434e950-3405-4e36-8a55-358945502763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:49.686062 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434e950-3405-4e36-8a55-358945502763 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:49.686354 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434e950-3405-4e36-8a55-358945502763 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:49.686649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434e950-3405-4e36-8a55-358945502763 None admin] PATH_INFO: `/v3/projects/22414db73cef49549c4930bec9f6b5dc` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:49.690987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5434e950-3405-4e36-8a55-358945502763 None admin] RBAC: Authorizing `identity:get_project(project_id=22414db73cef49549c4930bec9f6b5dc)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:49.692450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5434e950-3405-4e36-8a55-358945502763 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:49.694410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 182/374] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:49 2022] GET /identity/v3/projects/22414db73cef49549c4930bec9f6b5dc => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:49.700215 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5434e950-3405-4e36-8a55-358945502763 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:49.846065 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 193/375] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 00:34:49.861054 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84029a9-26fc-4c43-a716-5215cf231094 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:49.888393 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5434e950-3405-4e36-8a55-358945502763 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:49.944722 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67851215-566a-453d-8277-7a9cf8c6ed96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:49.945289 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67851215-566a-453d-8277-7a9cf8c6ed96 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:49.945614 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67851215-566a-453d-8277-7a9cf8c6ed96 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:49.945894 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67851215-566a-453d-8277-7a9cf8c6ed96 None admin] PATH_INFO: `/v3/groups/008b22bc962749f88dcab89d95b610bd` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:49.947027 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67851215-566a-453d-8277-7a9cf8c6ed96 None admin] RBAC: Authorizing `identity:get_group(group_id=008b22bc962749f88dcab89d95b610bd)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:49.948679 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67851215-566a-453d-8277-7a9cf8c6ed96 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:49.951165 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 183/376] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Aug 26 00:34:49 2022] GET /identity/v3/groups/008b22bc962749f88dcab89d95b610bd => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:49.961143 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67851215-566a-453d-8277-7a9cf8c6ed96 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:50.017479 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:50.023490 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84029a9-26fc-4c43-a716-5215cf231094 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:50.062853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee746d-cb36-4fdb-8844-0c759c5a3a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:50.064035 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee746d-cb36-4fdb-8844-0c759c5a3a57 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:50.064326 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee746d-cb36-4fdb-8844-0c759c5a3a57 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:50.064686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee746d-cb36-4fdb-8844-0c759c5a3a57 None admin] PATH_INFO: `/v3/projects/86ade54a72a44599b7a6313f909a3551` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:50.065905 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee746d-cb36-4fdb-8844-0c759c5a3a57 None admin] RBAC: Authorizing `identity:get_project(project_id=86ade54a72a44599b7a6313f909a3551)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:50.067392 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee746d-cb36-4fdb-8844-0c759c5a3a57 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:50.069163 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 194/377] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:49 2022] GET /identity/v3/projects/86ade54a72a44599b7a6313f909a3551 => generated 344 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:50.074393 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:50.074934 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee746d-cb36-4fdb-8844-0c759c5a3a57 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:50.157572 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67851215-566a-453d-8277-7a9cf8c6ed96 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:50.189482 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee746d-cb36-4fdb-8844-0c759c5a3a57 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:50.212585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67a1b31-df06-469f-915c-d03b1eced6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:50.212585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a1b31-df06-469f-915c-d03b1eced6ef None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:50.212585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a1b31-df06-469f-915c-d03b1eced6ef None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:50.212585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a1b31-df06-469f-915c-d03b1eced6ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:50.212934 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67a1b31-df06-469f-915c-d03b1eced6ef None admin] RBAC: Adding query filter params (group.id=008b22bc962749f88dcab89d95b610bd, role.id=423aa6ac8b05402da7587e83e5c4ce09, scope.project.id=22414db73cef49549c4930bec9f6b5dc) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:50.212934 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67a1b31-df06-469f-915c-d03b1eced6ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:50.214053 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67a1b31-df06-469f-915c-d03b1eced6ef None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:50.220766 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 184/378] 10.222.0.59 () {56 vars in 1376 bytes} [Fri Aug 26 00:34:49 2022] GET /identity/v3/role_assignments?group.id=008b22bc962749f88dcab89d95b610bd&scope.project.id=22414db73cef49549c4930bec9f6b5dc&role.id=423aa6ac8b05402da7587e83e5c4ce09 => generated 260 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:50.222069 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0b70a-1678-49a4-8bc1-c4828158bac3 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:50.222363 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0b70a-1678-49a4-8bc1-c4828158bac3 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:50.222643 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0b70a-1678-49a4-8bc1-c4828158bac3 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:50.224153 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 185/379] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:50.234102 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df46f5-bcb2-48cf-82cc-8cb958fbaf4d None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:50.234401 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df46f5-bcb2-48cf-82cc-8cb958fbaf4d None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:50.234661 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df46f5-bcb2-48cf-82cc-8cb958fbaf4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:50.241305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9d04a-9de7-47d2-9f8e-3f1e20d825ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:50.241962 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9d04a-9de7-47d2-9f8e-3f1e20d825ba None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:50.242242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9d04a-9de7-47d2-9f8e-3f1e20d825ba None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:50.242518 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9d04a-9de7-47d2-9f8e-3f1e20d825ba None admin] PATH_INFO: `/v3/groups/bf671cb739444d1798469f5fc7dfaa16` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:50.249634 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa9d04a-9de7-47d2-9f8e-3f1e20d825ba None admin] RBAC: Authorizing `identity:get_group(group_id=bf671cb739444d1798469f5fc7dfaa16)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:50.251315 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa9d04a-9de7-47d2-9f8e-3f1e20d825ba None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:50.253283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 195/380] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Aug 26 00:34:50 2022] GET /identity/v3/groups/bf671cb739444d1798469f5fc7dfaa16 => generated 229 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:50.272640 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-b3df46f5-bcb2-48cf-82cc-8cb958fbaf4d None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:50.272945 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9d04a-9de7-47d2-9f8e-3f1e20d825ba None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:50.273982 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-b3df46f5-bcb2-48cf-82cc-8cb958fbaf4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:50.320925 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:50.397622 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 186/381] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:34:50.418733 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9d04a-9de7-47d2-9f8e-3f1e20d825ba None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:50.456143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04aa91-1ded-45b5-b147-59c8c67bf894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:50.459003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04aa91-1ded-45b5-b147-59c8c67bf894 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:50.459235 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04aa91-1ded-45b5-b147-59c8c67bf894 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:50.459320 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04aa91-1ded-45b5-b147-59c8c67bf894 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:50.460051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04aa91-1ded-45b5-b147-59c8c67bf894 None admin] RBAC: Adding query filter params (group.id=bf671cb739444d1798469f5fc7dfaa16, role.id=1a33e3c8cb8347feb9498595886cc621, scope.project.id=86ade54a72a44599b7a6313f909a3551) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:50.460361 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04aa91-1ded-45b5-b147-59c8c67bf894 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:50.461855 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04aa91-1ded-45b5-b147-59c8c67bf894 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:50.468581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 196/382] 10.222.0.59 () {56 vars in 1376 bytes} [Fri Aug 26 00:34:50 2022] GET /identity/v3/role_assignments?group.id=bf671cb739444d1798469f5fc7dfaa16&scope.project.id=86ade54a72a44599b7a6313f909a3551&role.id=1a33e3c8cb8347feb9498595886cc621 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:50.717527 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df46f5-bcb2-48cf-82cc-8cb958fbaf4d None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:50.773853 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:50.829206 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df46f5-bcb2-48cf-82cc-8cb958fbaf4d None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:50.865404 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fcc39d-cb82-4a92-ad0d-4e583ad4135e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:50.866405 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcc39d-cb82-4a92-ad0d-4e583ad4135e None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:50.866888 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcc39d-cb82-4a92-ad0d-4e583ad4135e None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:50.867358 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcc39d-cb82-4a92-ad0d-4e583ad4135e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:50.882814 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fcc39d-cb82-4a92-ad0d-4e583ad4135e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:50.888291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-c7fcc39d-cb82-4a92-ad0d-4e583ad4135e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 00:34:50.888291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:50.888291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:50.888291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:50.888291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:50.888291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:50.888291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:50.888291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:50.888291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:50.888291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:50.888291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:50.888291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:50.888291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:50.888291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:50.888291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:50.888291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:50.889259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:50.889259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:50.889259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:50.889259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:50.889259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:50.889259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:50.889259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:50.889259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:50.889259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:50.889259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:50.889259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:50.889259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:50.889259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:50.889259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:50.889259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:50.889259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:50.890113 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:50.890113 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:50.890113 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:50.890113 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:50.890113 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:50.890113 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:50.890113 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:50.890113 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:50.890113 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:50.890113 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:50.890113 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 00:34:50.890113 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:34:50.892448 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 187/383] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Aug 26 00:34:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:50.898733 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04aa91-1ded-45b5-b147-59c8c67bf894 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:51.034411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:51.050709 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04aa91-1ded-45b5-b147-59c8c67bf894 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:51.092503 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8746f-a938-4a2c-9a85-054c38c3f6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:51.093124 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8746f-a938-4a2c-9a85-054c38c3f6eb None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:51.093404 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8746f-a938-4a2c-9a85-054c38c3f6eb None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:51.093684 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8746f-a938-4a2c-9a85-054c38c3f6eb None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:51.094381 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8746f-a938-4a2c-9a85-054c38c3f6eb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:51.094691 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8746f-a938-4a2c-9a85-054c38c3f6eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:51.096084 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8746f-a938-4a2c-9a85-054c38c3f6eb None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:51.108169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 197/384] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:34:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:51.114470 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fcc39d-cb82-4a92-ad0d-4e583ad4135e None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:51.152419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8a2ab-9550-4dd3-a4a6-ccd0a85a5cfb None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:51.152786 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8a2ab-9550-4dd3-a4a6-ccd0a85a5cfb None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:51.153010 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8a2ab-9550-4dd3-a4a6-ccd0a85a5cfb None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:51.154865 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 198/385] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:51.159895 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af5f71-c478-4b5f-a005-c9254058f088 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:51.160194 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af5f71-c478-4b5f-a005-c9254058f088 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:51.160492 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af5f71-c478-4b5f-a005-c9254058f088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:51.193182 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-44af5f71-c478-4b5f-a005-c9254058f088 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:51.193182 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-44af5f71-c478-4b5f-a005-c9254058f088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:51.285829 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fcc39d-cb82-4a92-ad0d-4e583ad4135e None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:51.288693 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 199/386] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:34:51.305405 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:51.344102 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d9a8d-db19-4f3a-bd66-00319d74cf96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:51.344795 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d9a8d-db19-4f3a-bd66-00319d74cf96 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:51.345079 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d9a8d-db19-4f3a-bd66-00319d74cf96 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:51.345356 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d9a8d-db19-4f3a-bd66-00319d74cf96 None admin] PATH_INFO: `/v3/groups/008b22bc962749f88dcab89d95b610bd` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:51.349126 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d9a8d-db19-4f3a-bd66-00319d74cf96 None admin] RBAC: Authorizing `identity:get_group(group_id=008b22bc962749f88dcab89d95b610bd)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:51.350581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d9a8d-db19-4f3a-bd66-00319d74cf96 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:51.352493 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 188/387] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Aug 26 00:34:51 2022] GET /identity/v3/groups/008b22bc962749f88dcab89d95b610bd => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:51.357655 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af5f71-c478-4b5f-a005-c9254058f088 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:51.454616 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:51.474326 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af5f71-c478-4b5f-a005-c9254058f088 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:51.517452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c40c37-f039-47d4-bcc0-8368c5316238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:51.518454 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c40c37-f039-47d4-bcc0-8368c5316238 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:51.518953 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c40c37-f039-47d4-bcc0-8368c5316238 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:51.519561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c40c37-f039-47d4-bcc0-8368c5316238 None admin] PATH_INFO: `/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:51.521296 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c40c37-f039-47d4-bcc0-8368c5316238 None admin] RBAC: Authorizing `identity:get_project(project_id=c60b6aa5591d4178ba9eb5fd05b7ee73)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:51.523040 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c40c37-f039-47d4-bcc0-8368c5316238 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:51.527087 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 200/388] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:51 2022] GET /identity/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:51.533342 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d9a8d-db19-4f3a-bd66-00319d74cf96 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:51.666748 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d9a8d-db19-4f3a-bd66-00319d74cf96 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:51.701406 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11161a93-7b48-40f5-b497-6bdc54b51954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:51.702069 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11161a93-7b48-40f5-b497-6bdc54b51954 None admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:51.702350 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11161a93-7b48-40f5-b497-6bdc54b51954 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:51.702627 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11161a93-7b48-40f5-b497-6bdc54b51954 None admin] PATH_INFO: `/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73/groups/008b22bc962749f88dcab89d95b610bd/roles/423aa6ac8b05402da7587e83e5c4ce09` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:51.704879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11161a93-7b48-40f5-b497-6bdc54b51954 None admin] RBAC: Authorizing `identity:create_grant(project_id=c60b6aa5591d4178ba9eb5fd05b7ee73, group_id=008b22bc962749f88dcab89d95b610bd, role_id=423aa6ac8b05402da7587e83e5c4ce09)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:51.706329 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11161a93-7b48-40f5-b497-6bdc54b51954 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:51.706770 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533768c-b20a-43a1-8429-936e5edcafad None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:51.707247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533768c-b20a-43a1-8429-936e5edcafad None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:51.707736 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533768c-b20a-43a1-8429-936e5edcafad None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:51.716500 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 201/389] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:51.716500 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0e1e5-0e5a-45c3-8d7f-55969db4a189 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:51.716500 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0e1e5-0e5a-45c3-8d7f-55969db4a189 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:51.716500 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0e1e5-0e5a-45c3-8d7f-55969db4a189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:51.733732 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-00e0e1e5-0e5a-45c3-8d7f-55969db4a189 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:51.735266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-00e0e1e5-0e5a-45c3-8d7f-55969db4a189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:51.801607 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 189/390] 10.222.0.59 () {58 vars in 1442 bytes} [Fri Aug 26 00:34:51 2022] PUT /identity/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73/groups/008b22bc962749f88dcab89d95b610bd/roles/423aa6ac8b05402da7587e83e5c4ce09 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:34:51.804974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11161a93-7b48-40f5-b497-6bdc54b51954 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:51.817482 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 202/391] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:51.872640 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:51.952045 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11161a93-7b48-40f5-b497-6bdc54b51954 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:51.995041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109168b6-8e59-473e-870c-d17eb7c07dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:51.995679 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109168b6-8e59-473e-870c-d17eb7c07dd4 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:51.995955 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109168b6-8e59-473e-870c-d17eb7c07dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:51.996224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109168b6-8e59-473e-870c-d17eb7c07dd4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:52.007271 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109168b6-8e59-473e-870c-d17eb7c07dd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:52.014513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-109168b6-8e59-473e-870c-d17eb7c07dd4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:52.014513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:52.014513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:52.014513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:52.014513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:52.014513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:52.014513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:52.014513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:52.014513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:52.014513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:52.014513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:52.014513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:52.014513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:52.014513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:52.014513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:52.014513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:52.015408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:52.015408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:52.015408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:52.015408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:52.015408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:52.015408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:52.015408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:52.015408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:52.015408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:52.015408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:52.015408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:52.015408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:52.015408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:52.015408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:52.015408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:52.015408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:52.016267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:52.016267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:52.016267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:52.016267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:52.016267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:52.016267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:52.016267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:52.016267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:52.016267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:52.016267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:52.016267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:52.016267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:34:52.016267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 190/392] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:52.021611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0e1e5-0e5a-45c3-8d7f-55969db4a189 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:52.108989 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:52.129513 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0e1e5-0e5a-45c3-8d7f-55969db4a189 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:52.173529 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14f3af-9c62-4a4c-88a2-b62264577c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:52.174150 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14f3af-9c62-4a4c-88a2-b62264577c0f None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:52.174427 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14f3af-9c62-4a4c-88a2-b62264577c0f None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:52.174701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14f3af-9c62-4a4c-88a2-b62264577c0f None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:52.175371 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14f3af-9c62-4a4c-88a2-b62264577c0f None admin] RBAC: Adding query filter params (name=member) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:52.175675 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14f3af-9c62-4a4c-88a2-b62264577c0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:52.177079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14f3af-9c62-4a4c-88a2-b62264577c0f None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:52.187879 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 203/393] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:34:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:52.194293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109168b6-8e59-473e-870c-d17eb7c07dd4 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:52.215488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14f3af-9c62-4a4c-88a2-b62264577c0f None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:52.354344 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:52.368519 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109168b6-8e59-473e-870c-d17eb7c07dd4 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:52.405898 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39637229-5a04-4239-a372-571b35800cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:52.408718 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39637229-5a04-4239-a372-571b35800cde None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:52.409324 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39637229-5a04-4239-a372-571b35800cde None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:52.409785 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39637229-5a04-4239-a372-571b35800cde None admin] PATH_INFO: `/v3/groups/008b22bc962749f88dcab89d95b610bd` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:52.411903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39637229-5a04-4239-a372-571b35800cde None admin] RBAC: Authorizing `identity:get_group(group_id=008b22bc962749f88dcab89d95b610bd)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:52.413605 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39637229-5a04-4239-a372-571b35800cde None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:52.415857 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 191/394] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Aug 26 00:34:52 2022] GET /identity/v3/groups/008b22bc962749f88dcab89d95b610bd => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:52.423815 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39637229-5a04-4239-a372-571b35800cde None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:52.428419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14f3af-9c62-4a4c-88a2-b62264577c0f None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:52.469815 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71969e96-d74b-4205-aa52-77e42534d5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:52.470382 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71969e96-d74b-4205-aa52-77e42534d5f5 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:52.470918 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71969e96-d74b-4205-aa52-77e42534d5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:52.471197 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71969e96-d74b-4205-aa52-77e42534d5f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:52.494085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71969e96-d74b-4205-aa52-77e42534d5f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:52.500135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-71969e96-d74b-4205-aa52-77e42534d5f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:52.500135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:52.500135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:52.500135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:52.500135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:52.500135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:52.500135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:52.500135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:52.500135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:52.500135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:52.500135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:52.500135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:52.500135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:52.500135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:52.500135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:52.500135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:52.501107 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:52.501107 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:52.501107 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:52.501107 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:52.501107 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:52.501107 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:52.501107 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:52.501107 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:52.501107 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:52.501107 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:52.501107 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:52.501107 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:52.501107 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:52.501107 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:52.501107 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:52.501107 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:52.501940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:52.501940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:52.501940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:52.501940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:52.501940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:52.501940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:52.501940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:52.501940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:52.501940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:52.501940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:52.501940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:52.501940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:34:52.513770 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 204/395] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:34:52.517705 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71969e96-d74b-4205-aa52-77e42534d5f5 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:52.528074 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:52.598932 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39637229-5a04-4239-a372-571b35800cde None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:52.644040 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71969e96-d74b-4205-aa52-77e42534d5f5 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:52.645620 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b66945-6420-4335-ab91-f081064cc5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:52.647405 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b66945-6420-4335-ab91-f081064cc5c8 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:52.647870 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b66945-6420-4335-ab91-f081064cc5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:52.648328 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b66945-6420-4335-ab91-f081064cc5c8 None admin] PATH_INFO: `/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:52.649692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b66945-6420-4335-ab91-f081064cc5c8 None admin] RBAC: Authorizing `identity:get_project(project_id=c60b6aa5591d4178ba9eb5fd05b7ee73)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:52.651989 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b66945-6420-4335-ab91-f081064cc5c8 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:52.655097 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 192/396] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:52 2022] GET /identity/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:52.657380 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8726024-55e5-4ce9-af32-1c02b1bfa712 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:52.660718 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8726024-55e5-4ce9-af32-1c02b1bfa712 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:52.660718 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8726024-55e5-4ce9-af32-1c02b1bfa712 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:52.660718 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 193/397] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:52.661435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8726024-55e5-4ce9-af32-1c02b1bfa712 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:52.691541 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a64ff-55c6-4222-97c8-958bb531c457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:52.692416 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a64ff-55c6-4222-97c8-958bb531c457 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:52.692731 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a64ff-55c6-4222-97c8-958bb531c457 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:52.693007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a64ff-55c6-4222-97c8-958bb531c457 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:52.694100 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a64ff-55c6-4222-97c8-958bb531c457 None admin] RBAC: Adding query filter params (name=member) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:52.694407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a64ff-55c6-4222-97c8-958bb531c457 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:52.696416 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a64ff-55c6-4222-97c8-958bb531c457 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:52.701561 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:52.712548 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 205/398] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:34:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:52.715896 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2220260-f60d-40bf-996d-768d0c555549 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:52.716182 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2220260-f60d-40bf-996d-768d0c555549 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:52.716569 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2220260-f60d-40bf-996d-768d0c555549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:52.742696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-e2220260-f60d-40bf-996d-768d0c555549 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:52.744804 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-e2220260-f60d-40bf-996d-768d0c555549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:52.784463 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8726024-55e5-4ce9-af32-1c02b1bfa712 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:52.821348 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:52.826549 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc97798-41a5-4320-b766-5f57ebfa4fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:52.827622 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc97798-41a5-4320-b766-5f57ebfa4fea None admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:52.827851 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc97798-41a5-4320-b766-5f57ebfa4fea None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:52.832089 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc97798-41a5-4320-b766-5f57ebfa4fea None admin] PATH_INFO: `/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73/groups/008b22bc962749f88dcab89d95b610bd/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:52.832089 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc97798-41a5-4320-b766-5f57ebfa4fea None admin] RBAC: Authorizing `identity:create_grant(project_id=c60b6aa5591d4178ba9eb5fd05b7ee73, group_id=008b22bc962749f88dcab89d95b610bd, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:52.832689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc97798-41a5-4320-b766-5f57ebfa4fea None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:52.840844 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 206/399] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:52.840844 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2220260-f60d-40bf-996d-768d0c555549 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:52.919198 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 194/400] 10.222.0.59 () {58 vars in 1442 bytes} [Fri Aug 26 00:34:52 2022] PUT /identity/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73/groups/008b22bc962749f88dcab89d95b610bd/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:34:52.968459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a60c5d-840f-43ec-8896-b3d5c4ec3a3a None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:52.968974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a60c5d-840f-43ec-8896-b3d5c4ec3a3a None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:52.969883 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a60c5d-840f-43ec-8896-b3d5c4ec3a3a None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:52.971559 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 195/401] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:34:52.974730 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2220260-f60d-40bf-996d-768d0c555549 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:52.976956 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619eb22-b5bc-43e2-b1d7-6bf281e68b48 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:52.977316 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619eb22-b5bc-43e2-b1d7-6bf281e68b48 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:52.977597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619eb22-b5bc-43e2-b1d7-6bf281e68b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:53.027016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-6619eb22-b5bc-43e2-b1d7-6bf281e68b48 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:53.027016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-6619eb22-b5bc-43e2-b1d7-6bf281e68b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:53.027295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c32210-42bb-4333-8815-69b1b8647d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:53.027295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c32210-42bb-4333-8815-69b1b8647d60 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:53.027295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c32210-42bb-4333-8815-69b1b8647d60 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:53.027295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c32210-42bb-4333-8815-69b1b8647d60 None admin] PATH_INFO: `/v3/groups/008b22bc962749f88dcab89d95b610bd` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:53.027613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c32210-42bb-4333-8815-69b1b8647d60 None admin] RBAC: Authorizing `identity:get_group(group_id=008b22bc962749f88dcab89d95b610bd)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:53.029879 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c32210-42bb-4333-8815-69b1b8647d60 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:53.032337 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 207/402] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Aug 26 00:34:52 2022] GET /identity/v3/groups/008b22bc962749f88dcab89d95b610bd => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:53.037981 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c32210-42bb-4333-8815-69b1b8647d60 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:53.131868 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 196/403] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:53.154763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:53.169267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619eb22-b5bc-43e2-b1d7-6bf281e68b48 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:53.201589 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c32210-42bb-4333-8815-69b1b8647d60 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:53.204870 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:53.244327 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ed249-f438-4645-a72e-535c70d3216c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:53.245663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ed249-f438-4645-a72e-535c70d3216c None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:53.246161 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ed249-f438-4645-a72e-535c70d3216c None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:53.246637 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ed249-f438-4645-a72e-535c70d3216c None admin] PATH_INFO: `/v3/projects/22414db73cef49549c4930bec9f6b5dc` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:53.248135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91ed249-f438-4645-a72e-535c70d3216c None admin] RBAC: Authorizing `identity:get_project(project_id=22414db73cef49549c4930bec9f6b5dc)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:53.250963 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91ed249-f438-4645-a72e-535c70d3216c None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:53.252834 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 208/404] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:53 2022] GET /identity/v3/projects/22414db73cef49549c4930bec9f6b5dc => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:53.261373 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ed249-f438-4645-a72e-535c70d3216c None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:53.306011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619eb22-b5bc-43e2-b1d7-6bf281e68b48 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:53.372460 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97f445-f683-4337-96ec-9b09f4574076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:53.373232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97f445-f683-4337-96ec-9b09f4574076 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:53.373397 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97f445-f683-4337-96ec-9b09f4574076 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:53.373678 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97f445-f683-4337-96ec-9b09f4574076 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:53.399064 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97f445-f683-4337-96ec-9b09f4574076 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:53.406767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-0c97f445-f683-4337-96ec-9b09f4574076 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 00:34:53.406767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:53.406767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:53.406767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:53.406767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:53.406767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:53.406767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:53.406767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:53.406767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:53.406767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:53.406767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:53.406767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:53.406767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:53.406767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:53.406767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:53.406767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:53.407767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:53.407767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:53.407767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:53.407767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:53.407767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:53.407767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:53.407767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:53.407767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:53.407767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:53.407767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:53.407767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:53.407767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:53.407767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:53.407767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:53.407767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:53.407767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:53.409144 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:53.409144 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:53.409144 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:53.409144 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:53.409144 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:53.409144 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:53.409144 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:53.409144 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:53.409144 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:53.409144 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:53.409144 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 00:34:53.409144 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:34:53.409144 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 197/405] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Aug 26 00:34:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:34:53.413498 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97f445-f683-4337-96ec-9b09f4574076 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:53.459806 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ed249-f438-4645-a72e-535c70d3216c None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:53.489332 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:53.496005 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8a8e8-78c5-4e51-92f6-c206358f7e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:53.502124 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8a8e8-78c5-4e51-92f6-c206358f7e46 None admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:53.502124 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8a8e8-78c5-4e51-92f6-c206358f7e46 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:53.502124 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8a8e8-78c5-4e51-92f6-c206358f7e46 None admin] PATH_INFO: `/v3/projects/22414db73cef49549c4930bec9f6b5dc/groups/008b22bc962749f88dcab89d95b610bd/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:53.502124 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8a8e8-78c5-4e51-92f6-c206358f7e46 None admin] RBAC: Authorizing `identity:create_grant(project_id=22414db73cef49549c4930bec9f6b5dc, group_id=008b22bc962749f88dcab89d95b610bd, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:53.502124 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8a8e8-78c5-4e51-92f6-c206358f7e46 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:53.523641 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:53.538449 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97f445-f683-4337-96ec-9b09f4574076 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:53.574587 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd6417-6d90-41b8-934d-1143577cbcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:53.575198 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd6417-6d90-41b8-934d-1143577cbcdd None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:53.575476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd6417-6d90-41b8-934d-1143577cbcdd None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:53.575748 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd6417-6d90-41b8-934d-1143577cbcdd None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:53.576409 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd6417-6d90-41b8-934d-1143577cbcdd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:53.576733 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd6417-6d90-41b8-934d-1143577cbcdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:53.578143 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd6417-6d90-41b8-934d-1143577cbcdd None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:53.585037 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 209/406] 10.222.0.59 () {58 vars in 1442 bytes} [Fri Aug 26 00:34:53 2022] PUT /identity/v3/projects/22414db73cef49549c4930bec9f6b5dc/groups/008b22bc962749f88dcab89d95b610bd/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:34:53.586333 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8a8e8-78c5-4e51-92f6-c206358f7e46 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:53.586443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 198/407] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:34:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:53.598407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd6417-6d90-41b8-934d-1143577cbcdd None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:53.725319 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8a8e8-78c5-4e51-92f6-c206358f7e46 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:53.773179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14588bde-60fa-4e52-a327-0520fafff496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:53.774603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14588bde-60fa-4e52-a327-0520fafff496 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:53.775339 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14588bde-60fa-4e52-a327-0520fafff496 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:53.775820 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14588bde-60fa-4e52-a327-0520fafff496 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:53.783637 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd6417-6d90-41b8-934d-1143577cbcdd None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:53.790790 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14588bde-60fa-4e52-a327-0520fafff496 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:53.798221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-14588bde-60fa-4e52-a327-0520fafff496 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:34:53.798221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:53.798221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:53.798221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:53.798221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:53.798221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:53.798221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:53.798221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:53.798221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:53.798221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:53.798221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:53.798221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:53.798221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:53.798221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:53.798221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:53.798221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:53.799123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:53.799123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:53.799123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:53.799123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:53.799123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:53.799123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:53.799123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:53.799123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:53.799123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:53.799123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:53.799123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:53.799123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:53.799123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:53.799123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:53.799123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:53.799123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:53.799944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:53.799944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:53.799944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:53.799944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:53.799944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:53.799944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:53.799944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:53.799944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:53.799944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:53.799944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:53.799944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:34:53.799944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:34:53.804653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 210/408] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:34:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:53.808322 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14588bde-60fa-4e52-a327-0520fafff496 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:53.832174 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74add295-0a23-4e9b-ac02-1af94c4b082c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:53.832768 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74add295-0a23-4e9b-ac02-1af94c4b082c None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:53.833223 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74add295-0a23-4e9b-ac02-1af94c4b082c None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:53.833498 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74add295-0a23-4e9b-ac02-1af94c4b082c None admin] PATH_INFO: `/v3/groups/008b22bc962749f88dcab89d95b610bd` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:53.835568 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74add295-0a23-4e9b-ac02-1af94c4b082c None admin] RBAC: Authorizing `identity:get_group(group_id=008b22bc962749f88dcab89d95b610bd)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:53.837034 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74add295-0a23-4e9b-ac02-1af94c4b082c None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:53.845812 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 199/409] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Aug 26 00:34:53 2022] GET /identity/v3/groups/008b22bc962749f88dcab89d95b610bd => generated 226 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:53.851082 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74add295-0a23-4e9b-ac02-1af94c4b082c None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:53.878473 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:53.901550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:53.996087 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14588bde-60fa-4e52-a327-0520fafff496 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:54.032085 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74add295-0a23-4e9b-ac02-1af94c4b082c None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:54.057636 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f8af6-ddac-4e12-b39e-8b8b544f2560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:54.058258 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f8af6-ddac-4e12-b39e-8b8b544f2560 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:54.058542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f8af6-ddac-4e12-b39e-8b8b544f2560 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:54.058818 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f8af6-ddac-4e12-b39e-8b8b544f2560 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:54.059494 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f8af6-ddac-4e12-b39e-8b8b544f2560 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:54.059795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f8af6-ddac-4e12-b39e-8b8b544f2560 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:54.061999 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f8af6-ddac-4e12-b39e-8b8b544f2560 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:54.081357 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 211/410] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:54.087948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f8af6-ddac-4e12-b39e-8b8b544f2560 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:54.105398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada23c3f-14b1-4079-9904-ee5f593ab7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:54.106043 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada23c3f-14b1-4079-9904-ee5f593ab7e8 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:54.106321 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada23c3f-14b1-4079-9904-ee5f593ab7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:54.106656 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada23c3f-14b1-4079-9904-ee5f593ab7e8 None admin] PATH_INFO: `/v3/projects/22414db73cef49549c4930bec9f6b5dc` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:54.108176 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada23c3f-14b1-4079-9904-ee5f593ab7e8 None admin] RBAC: Authorizing `identity:get_project(project_id=22414db73cef49549c4930bec9f6b5dc)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:54.112099 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada23c3f-14b1-4079-9904-ee5f593ab7e8 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:54.120657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 200/411] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:53 2022] GET /identity/v3/projects/22414db73cef49549c4930bec9f6b5dc => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:54.123666 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada23c3f-14b1-4079-9904-ee5f593ab7e8 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:54.248314 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f8af6-ddac-4e12-b39e-8b8b544f2560 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:54.256671 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:54.261570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:54.276282 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada23c3f-14b1-4079-9904-ee5f593ab7e8 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:54.284347 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9dd8b7-51c8-461a-97e2-85cf07f9c4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:54.285206 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9dd8b7-51c8-461a-97e2-85cf07f9c4b5 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:54.285645 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9dd8b7-51c8-461a-97e2-85cf07f9c4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:54.286072 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9dd8b7-51c8-461a-97e2-85cf07f9c4b5 None admin] PATH_INFO: `/v3/groups/bf671cb739444d1798469f5fc7dfaa16` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:54.287324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9dd8b7-51c8-461a-97e2-85cf07f9c4b5 None admin] RBAC: Authorizing `identity:get_group(group_id=bf671cb739444d1798469f5fc7dfaa16)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:54.289096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9dd8b7-51c8-461a-97e2-85cf07f9c4b5 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:54.291107 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 212/412] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Aug 26 00:34:54 2022] GET /identity/v3/groups/bf671cb739444d1798469f5fc7dfaa16 => generated 229 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:54.292499 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92579f-dd57-4389-a066-389bfcdd3ef7 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:54.292958 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92579f-dd57-4389-a066-389bfcdd3ef7 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:54.293404 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92579f-dd57-4389-a066-389bfcdd3ef7 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:54.299949 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 213/413] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:54.301369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cac01-67ab-4608-bbdf-42c75c4659b5 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:54.301861 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cac01-67ab-4608-bbdf-42c75c4659b5 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:54.302303 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cac01-67ab-4608-bbdf-42c75c4659b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:54.313999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2e8cf-edd3-4b47-bec0-94552453a485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:54.314667 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2e8cf-edd3-4b47-bec0-94552453a485 None admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:54.314948 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2e8cf-edd3-4b47-bec0-94552453a485 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:54.315224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2e8cf-edd3-4b47-bec0-94552453a485 None admin] PATH_INFO: `/v3/projects/22414db73cef49549c4930bec9f6b5dc/groups/008b22bc962749f88dcab89d95b610bd/roles/423aa6ac8b05402da7587e83e5c4ce09` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:54.317946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2e8cf-edd3-4b47-bec0-94552453a485 None admin] RBAC: Authorizing `identity:create_grant(project_id=22414db73cef49549c4930bec9f6b5dc, group_id=008b22bc962749f88dcab89d95b610bd, role_id=423aa6ac8b05402da7587e83e5c4ce09)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:54.319397 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2e8cf-edd3-4b47-bec0-94552453a485 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:54.321779 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-1c7cac01-67ab-4608-bbdf-42c75c4659b5 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:54.323180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-1c7cac01-67ab-4608-bbdf-42c75c4659b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:54.391577 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 201/414] 10.222.0.59 () {58 vars in 1442 bytes} [Fri Aug 26 00:34:54 2022] PUT /identity/v3/projects/22414db73cef49549c4930bec9f6b5dc/groups/008b22bc962749f88dcab89d95b610bd/roles/423aa6ac8b05402da7587e83e5c4ce09 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:34:54.397210 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2e8cf-edd3-4b47-bec0-94552453a485 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:54.407906 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 214/415] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:54.550832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2e8cf-edd3-4b47-bec0-94552453a485 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:54.592196 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33cf92-7700-4e21-8bb6-8cd99e576eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:54.592981 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33cf92-7700-4e21-8bb6-8cd99e576eee None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:54.593328 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33cf92-7700-4e21-8bb6-8cd99e576eee None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:54.593663 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33cf92-7700-4e21-8bb6-8cd99e576eee None admin] PATH_INFO: `/v3/projects/86ade54a72a44599b7a6313f909a3551` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:54.595361 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33cf92-7700-4e21-8bb6-8cd99e576eee None admin] RBAC: Authorizing `identity:get_project(project_id=86ade54a72a44599b7a6313f909a3551)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:54.596892 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33cf92-7700-4e21-8bb6-8cd99e576eee None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:54.599001 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 202/416] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:54 2022] GET /identity/v3/projects/86ade54a72a44599b7a6313f909a3551 => generated 344 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:54.604619 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7cac01-67ab-4608-bbdf-42c75c4659b5 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:54.709568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7cac01-67ab-4608-bbdf-42c75c4659b5 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:54.721254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:54.744797 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf4663-c3e4-4f83-8294-254389c99721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:54.745571 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf4663-c3e4-4f83-8294-254389c99721 None admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:54.745905 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf4663-c3e4-4f83-8294-254389c99721 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:54.755522 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf4663-c3e4-4f83-8294-254389c99721 None admin] PATH_INFO: `/v3/projects/86ade54a72a44599b7a6313f909a3551/groups/bf671cb739444d1798469f5fc7dfaa16/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:54.755522 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf4663-c3e4-4f83-8294-254389c99721 None admin] RBAC: Authorizing `identity:create_grant(project_id=86ade54a72a44599b7a6313f909a3551, group_id=bf671cb739444d1798469f5fc7dfaa16, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:54.755522 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf4663-c3e4-4f83-8294-254389c99721 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:54.832881 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 215/417] 10.222.0.59 () {58 vars in 1442 bytes} [Fri Aug 26 00:34:54 2022] PUT /identity/v3/projects/86ade54a72a44599b7a6313f909a3551/groups/bf671cb739444d1798469f5fc7dfaa16/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:34:54.844985 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33cf92-7700-4e21-8bb6-8cd99e576eee None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:54.859054 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:54.963494 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33cf92-7700-4e21-8bb6-8cd99e576eee None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:55.003212 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab3ea1-7660-471c-ad1c-04b5018772cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:55.003871 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab3ea1-7660-471c-ad1c-04b5018772cf None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:55.004148 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab3ea1-7660-471c-ad1c-04b5018772cf None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:55.004525 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab3ea1-7660-471c-ad1c-04b5018772cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:55.017190 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab3ea1-7660-471c-ad1c-04b5018772cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:55.022201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-adab3ea1-7660-471c-ad1c-04b5018772cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 00:34:55.022201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:55.022201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:55.022201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:55.022201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:55.022201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:55.022201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:55.022201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:55.022201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:55.022201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:55.022201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:55.022201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:55.022201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:55.022201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:55.022201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:55.022201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:55.023154 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:55.023154 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:55.023154 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:55.023154 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:55.023154 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:55.023154 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:55.023154 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:55.023154 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:55.023154 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:55.023154 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:55.023154 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:55.023154 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:55.023154 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:55.023154 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:55.023154 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:55.023154 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:55.023958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:55.023958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:55.023958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:55.023958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:55.023958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:55.023958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:55.023958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:55.023958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:55.023958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:55.023958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:55.023958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 00:34:55.023958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:34:55.023958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 203/418] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Aug 26 00:34:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:55.032581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf4663-c3e4-4f83-8294-254389c99721 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:55.073735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef333d-4634-4747-abb1-c87a22091bce None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:55.074056 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef333d-4634-4747-abb1-c87a22091bce None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:55.074301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef333d-4634-4747-abb1-c87a22091bce None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:55.075808 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 204/419] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:55.086328 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565de66-eb3e-41db-83ff-df73d055fcfd None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:55.087041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565de66-eb3e-41db-83ff-df73d055fcfd None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:55.087578 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565de66-eb3e-41db-83ff-df73d055fcfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:55.115260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-9565de66-eb3e-41db-83ff-df73d055fcfd None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:55.116753 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-9565de66-eb3e-41db-83ff-df73d055fcfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:55.184580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf4663-c3e4-4f83-8294-254389c99721 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:55.218274 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:55.227890 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 205/420] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:34:55.239253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99066a7-1f2d-4e44-bd36-29fde0718ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:55.239876 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99066a7-1f2d-4e44-bd36-29fde0718ce7 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:55.240158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99066a7-1f2d-4e44-bd36-29fde0718ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:55.240433 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99066a7-1f2d-4e44-bd36-29fde0718ce7 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:55.241136 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99066a7-1f2d-4e44-bd36-29fde0718ce7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:55.241624 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99066a7-1f2d-4e44-bd36-29fde0718ce7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:55.243018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99066a7-1f2d-4e44-bd36-29fde0718ce7 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:55.252729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 216/421] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:34:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:55.258636 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9565de66-eb3e-41db-83ff-df73d055fcfd None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:55.374507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9565de66-eb3e-41db-83ff-df73d055fcfd None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:55.436670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f15699-ad7c-493c-a723-775fd1e9d1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:55.437300 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f15699-ad7c-493c-a723-775fd1e9d1a5 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:55.437583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f15699-ad7c-493c-a723-775fd1e9d1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:55.437861 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f15699-ad7c-493c-a723-775fd1e9d1a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:55.523507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:55.526491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f15699-ad7c-493c-a723-775fd1e9d1a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:55.527946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f15699-ad7c-493c-a723-775fd1e9d1a5 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:55.625397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99066a7-1f2d-4e44-bd36-29fde0718ce7 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:55.640151 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 206/422] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 00:34:55.645721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f15699-ad7c-493c-a723-775fd1e9d1a5 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:55.670953 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:55.754500 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:55.758475 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f15699-ad7c-493c-a723-775fd1e9d1a5 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:55.768459 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99066a7-1f2d-4e44-bd36-29fde0718ce7 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:55.815465 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e405a5-91cb-4fed-a7fb-a6b4ee47da97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:55.816439 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e405a5-91cb-4fed-a7fb-a6b4ee47da97 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:55.816933 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e405a5-91cb-4fed-a7fb-a6b4ee47da97 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:55.817393 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e405a5-91cb-4fed-a7fb-a6b4ee47da97 None admin] PATH_INFO: `/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:55.819396 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e405a5-91cb-4fed-a7fb-a6b4ee47da97 None admin] RBAC: Authorizing `identity:get_project(project_id=c60b6aa5591d4178ba9eb5fd05b7ee73)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:55.821911 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e405a5-91cb-4fed-a7fb-a6b4ee47da97 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:55.823919 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 207/423] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:55 2022] GET /identity/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:55.829490 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e405a5-91cb-4fed-a7fb-a6b4ee47da97 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:55.834533 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b3b314-677f-42ce-b829-d551f3113528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:55.836061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3b314-677f-42ce-b829-d551f3113528 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:55.836764 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3b314-677f-42ce-b829-d551f3113528 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:55.837237 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3b314-677f-42ce-b829-d551f3113528 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:55.857769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b3b314-677f-42ce-b829-d551f3113528 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:55.865853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-32b3b314-677f-42ce-b829-d551f3113528 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:55.865853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:55.865853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:55.865853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:55.865853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:55.865853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:55.865853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:55.865853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:55.865853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:55.865853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:55.865853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:55.865853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:55.865853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:55.865853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:55.865853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:55.865853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:55.866767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:55.866767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:55.866767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:55.866767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:55.866767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:55.866767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:55.866767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:55.866767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:55.866767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:55.866767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:55.866767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:55.866767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:55.866767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:55.866767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:55.866767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:55.866767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:55.867562 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:55.867562 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:55.867562 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:55.867562 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:55.867562 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:55.867562 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:55.867562 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:55.867562 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:55.867562 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:55.867562 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:55.867562 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:55.867562 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:34:55.869940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 217/424] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:34:55.877073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b3b314-677f-42ce-b829-d551f3113528 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:55.956648 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e405a5-91cb-4fed-a7fb-a6b4ee47da97 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:55.993520 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c07ee0-3e04-4671-a612-56cc62c20717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:55.994566 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c07ee0-3e04-4671-a612-56cc62c20717 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:55.995047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c07ee0-3e04-4671-a612-56cc62c20717 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:55.995514 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c07ee0-3e04-4671-a612-56cc62c20717 None admin] PATH_INFO: `/v3/groups/008b22bc962749f88dcab89d95b610bd` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:55.997007 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c07ee0-3e04-4671-a612-56cc62c20717 None admin] RBAC: Authorizing `identity:get_group(group_id=008b22bc962749f88dcab89d95b610bd)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:55.998826 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c07ee0-3e04-4671-a612-56cc62c20717 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:56.001119 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 208/425] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Aug 26 00:34:55 2022] GET /identity/v3/groups/008b22bc962749f88dcab89d95b610bd => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:56.009623 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c07ee0-3e04-4671-a612-56cc62c20717 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:56.063995 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:56.079041 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:56.087630 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b3b314-677f-42ce-b829-d551f3113528 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:56.126599 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d69a98-c61e-4a05-99f7-8f7293801bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:56.127377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d69a98-c61e-4a05-99f7-8f7293801bd3 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:56.127658 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d69a98-c61e-4a05-99f7-8f7293801bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:56.127983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d69a98-c61e-4a05-99f7-8f7293801bd3 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:56.128692 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d69a98-c61e-4a05-99f7-8f7293801bd3 None admin] RBAC: Adding query filter params (name=member) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:56.128998 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d69a98-c61e-4a05-99f7-8f7293801bd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:56.130593 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d69a98-c61e-4a05-99f7-8f7293801bd3 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:56.139500 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 218/426] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:34:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:56.140266 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c07ee0-3e04-4671-a612-56cc62c20717 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:56.140810 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c7d64-1cc1-41d7-bf17-76bec64a8cce None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:56.141130 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c7d64-1cc1-41d7-bf17-76bec64a8cce None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:56.141407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c7d64-1cc1-41d7-bf17-76bec64a8cce None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:56.146875 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 219/427] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:56.146875 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c7d64-1cc1-41d7-bf17-76bec64a8cce None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:56.188790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686bd656-5898-4655-8e8f-0eb71385bd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:56.189631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686bd656-5898-4655-8e8f-0eb71385bd07 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:56.190079 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686bd656-5898-4655-8e8f-0eb71385bd07 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:56.190727 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686bd656-5898-4655-8e8f-0eb71385bd07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:56.191611 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686bd656-5898-4655-8e8f-0eb71385bd07 None admin] RBAC: Adding query filter params (group.id=008b22bc962749f88dcab89d95b610bd, role.id=423aa6ac8b05402da7587e83e5c4ce09, scope.project.id=c60b6aa5591d4178ba9eb5fd05b7ee73) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:56.192055 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686bd656-5898-4655-8e8f-0eb71385bd07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:56.194156 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686bd656-5898-4655-8e8f-0eb71385bd07 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:56.202701 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 209/428] 10.222.0.59 () {56 vars in 1376 bytes} [Fri Aug 26 00:34:56 2022] GET /identity/v3/role_assignments?group.id=008b22bc962749f88dcab89d95b610bd&scope.project.id=c60b6aa5591d4178ba9eb5fd05b7ee73&role.id=423aa6ac8b05402da7587e83e5c4ce09 => generated 611 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:56.204486 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70b137-35dc-410e-aad0-ee94998f7511 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:56.204938 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70b137-35dc-410e-aad0-ee94998f7511 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:56.205353 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70b137-35dc-410e-aad0-ee94998f7511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:56.230675 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-dd70b137-35dc-410e-aad0-ee94998f7511 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:56.232659 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-dd70b137-35dc-410e-aad0-ee94998f7511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:56.260147 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c7d64-1cc1-41d7-bf17-76bec64a8cce None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:56.334492 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5312ed0-7d0d-493f-ae81-e3f375574845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:56.335111 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5312ed0-7d0d-493f-ae81-e3f375574845 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:56.335440 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5312ed0-7d0d-493f-ae81-e3f375574845 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:56.335720 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5312ed0-7d0d-493f-ae81-e3f375574845 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:56.363491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 210/429] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:56.396897 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:56.441164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5312ed0-7d0d-493f-ae81-e3f375574845 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:56.442726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5312ed0-7d0d-493f-ae81-e3f375574845 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:56.544757 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 220/430] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 00:34:56.552850 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd70b137-35dc-410e-aad0-ee94998f7511 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:56.584086 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:56.662915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd70b137-35dc-410e-aad0-ee94998f7511 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:56.702497 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b19942-2eb8-40b1-8f39-ade270f26f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:56.703131 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b19942-2eb8-40b1-8f39-ade270f26f5e None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:56.703413 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b19942-2eb8-40b1-8f39-ade270f26f5e None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:56.704249 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b19942-2eb8-40b1-8f39-ade270f26f5e None admin] PATH_INFO: `/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:56.705419 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b19942-2eb8-40b1-8f39-ade270f26f5e None admin] RBAC: Authorizing `identity:get_project(project_id=c60b6aa5591d4178ba9eb5fd05b7ee73)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:56.706840 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b19942-2eb8-40b1-8f39-ade270f26f5e None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:56.708805 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 211/431] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:56 2022] GET /identity/v3/projects/c60b6aa5591d4178ba9eb5fd05b7ee73 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:56.714042 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5312ed0-7d0d-493f-ae81-e3f375574845 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:56.725025 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b19942-2eb8-40b1-8f39-ade270f26f5e None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:56.786976 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:56.843185 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:56.844450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5312ed0-7d0d-493f-ae81-e3f375574845 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:56.848141 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b19942-2eb8-40b1-8f39-ade270f26f5e None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:56.885656 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1dbcb6-776e-499c-89c6-a85de3599443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:56.886329 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1dbcb6-776e-499c-89c6-a85de3599443 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:56.886575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1dbcb6-776e-499c-89c6-a85de3599443 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:56.886850 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1dbcb6-776e-499c-89c6-a85de3599443 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:56.888190 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f403c-d65c-4c83-a84c-1010db15c1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:56.889244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f403c-d65c-4c83-a84c-1010db15c1fa None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:56.890045 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f403c-d65c-4c83-a84c-1010db15c1fa None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:56.890532 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f403c-d65c-4c83-a84c-1010db15c1fa None admin] PATH_INFO: `/v3/groups/008b22bc962749f88dcab89d95b610bd` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:56.892755 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f403c-d65c-4c83-a84c-1010db15c1fa None admin] RBAC: Authorizing `identity:get_group(group_id=008b22bc962749f88dcab89d95b610bd)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:56.894760 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f403c-d65c-4c83-a84c-1010db15c1fa None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:56.897113 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 221/432] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Aug 26 00:34:56 2022] GET /identity/v3/groups/008b22bc962749f88dcab89d95b610bd => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:56.901346 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1dbcb6-776e-499c-89c6-a85de3599443 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:56.903163 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f403c-d65c-4c83-a84c-1010db15c1fa None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:56.908157 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-6a1dbcb6-776e-499c-89c6-a85de3599443 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:56.908157 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:56.908157 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:56.908157 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:56.908157 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:56.908157 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:56.908157 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:56.908157 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:56.908157 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:56.908157 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:56.908157 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:56.908157 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:56.908157 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:56.908157 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:56.908157 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:56.908157 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:56.909241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:56.909241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:56.909241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:56.909241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:56.909241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:56.909241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:56.909241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:56.909241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:56.909241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:56.909241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:56.909241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:56.909241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:56.909241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:56.909241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:56.909241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:56.909241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:56.910066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:56.910066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:56.910066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:56.910066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:56.910066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:56.910066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:56.910066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:56.910066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:56.910066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:56.910066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:56.910066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 00:34:56.910066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:34:56.916611 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 212/433] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:34:56.920406 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1dbcb6-776e-499c-89c6-a85de3599443 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:57.013215 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f403c-d65c-4c83-a84c-1010db15c1fa None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:57.030871 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1dbcb6-776e-499c-89c6-a85de3599443 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:57.036668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:57.051839 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5c95a-6438-4d44-b949-b279d445120f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:57.052463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5c95a-6438-4d44-b949-b279d445120f None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:57.052793 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5c95a-6438-4d44-b949-b279d445120f None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:57.053073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5c95a-6438-4d44-b949-b279d445120f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:57.053801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5c95a-6438-4d44-b949-b279d445120f None admin] RBAC: Adding query filter params (group.id=008b22bc962749f88dcab89d95b610bd, role.id=5032636d8729461c9926a59b9bcd041b, scope.project.id=c60b6aa5591d4178ba9eb5fd05b7ee73) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:57.054124 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5c95a-6438-4d44-b949-b279d445120f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:57.055604 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5c95a-6438-4d44-b949-b279d445120f None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:57.061386 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 222/434] 10.222.0.59 () {56 vars in 1376 bytes} [Fri Aug 26 00:34:56 2022] GET /identity/v3/role_assignments?group.id=008b22bc962749f88dcab89d95b610bd&scope.project.id=c60b6aa5591d4178ba9eb5fd05b7ee73&role.id=5032636d8729461c9926a59b9bcd041b => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:57.062606 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442ebd6-3e98-484b-8e67-33bd856e69f1 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:57.062937 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442ebd6-3e98-484b-8e67-33bd856e69f1 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:57.063721 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442ebd6-3e98-484b-8e67-33bd856e69f1 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:57.068512 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 223/435] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:57.071439 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a02ddf-7f09-47f3-a8c2-3f32a031445b None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:57.072762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a02ddf-7f09-47f3-a8c2-3f32a031445b None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:57.073635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a02ddf-7f09-47f3-a8c2-3f32a031445b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:57.083096 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb0ab2-4d4c-4de7-a50c-27889d5b700f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:57.084071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb0ab2-4d4c-4de7-a50c-27889d5b700f None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:57.084567 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb0ab2-4d4c-4de7-a50c-27889d5b700f None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:57.085018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb0ab2-4d4c-4de7-a50c-27889d5b700f None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:57.085922 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb0ab2-4d4c-4de7-a50c-27889d5b700f None admin] RBAC: Adding query filter params (name=member) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:57.086408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb0ab2-4d4c-4de7-a50c-27889d5b700f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:57.088108 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb0ab2-4d4c-4de7-a50c-27889d5b700f None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:57.100637 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 213/436] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:34:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:57.104231 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-77a02ddf-7f09-47f3-a8c2-3f32a031445b None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:57.105496 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-77a02ddf-7f09-47f3-a8c2-3f32a031445b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:57.108545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb0ab2-4d4c-4de7-a50c-27889d5b700f None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:57.138657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:57.194801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 224/437] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:34:57.196079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4d427-9111-492f-99ac-7ddd9a210146 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:57.196373 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4d427-9111-492f-99ac-7ddd9a210146 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:57.196678 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4d427-9111-492f-99ac-7ddd9a210146 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:57.198123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 225/438] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:34:57.203711 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06705845-39cb-4319-be39-b83aadc360bc None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:57.203988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06705845-39cb-4319-be39-b83aadc360bc None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:57.204270 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06705845-39cb-4319-be39-b83aadc360bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:57.225806 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-06705845-39cb-4319-be39-b83aadc360bc None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:34:57.227022 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-06705845-39cb-4319-be39-b83aadc360bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:34:57.250254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb0ab2-4d4c-4de7-a50c-27889d5b700f None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:57.306202 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88178d21-afab-403d-8d84-2cbc05a641f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:57.306403 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88178d21-afab-403d-8d84-2cbc05a641f5 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:57.306616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88178d21-afab-403d-8d84-2cbc05a641f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:57.306886 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88178d21-afab-403d-8d84-2cbc05a641f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:57.324235 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 226/439] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:34:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:34:57.386545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88178d21-afab-403d-8d84-2cbc05a641f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:57.388048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88178d21-afab-403d-8d84-2cbc05a641f5 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:57.411165 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:57.472826 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06705845-39cb-4319-be39-b83aadc360bc None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:57.502675 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 214/440] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:34:57.508494 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88178d21-afab-403d-8d84-2cbc05a641f5 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:57.531941 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:57.597722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06705845-39cb-4319-be39-b83aadc360bc None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:57.636930 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88178d21-afab-403d-8d84-2cbc05a641f5 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:57.638140 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb0789-5341-40e9-96b3-2ac43707970e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:57.638951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb0789-5341-40e9-96b3-2ac43707970e None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:57.640043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb0789-5341-40e9-96b3-2ac43707970e None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:57.640321 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb0789-5341-40e9-96b3-2ac43707970e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:57.652649 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfb0789-5341-40e9-96b3-2ac43707970e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:57.657729 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:57.657864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-acfb0789-5341-40e9-96b3-2ac43707970e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 00:34:57.657864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:57.657864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:57.657864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:57.657864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:57.657864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:57.657864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:57.657864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:57.657864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:57.657864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:57.657864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:57.657864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:57.657864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:57.657864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:57.657864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:57.657864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:57.658729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:57.658729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:57.658729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:57.658729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:57.658729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:57.658729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:57.658729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:57.658729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:57.658729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:57.658729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:57.658729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:57.658729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:57.658729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:57.658729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:57.658729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:57.658729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:57.659515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:57.659515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:57.659515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:57.659515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:57.659515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:57.659515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:57.659515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:57.659515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:57.659515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:57.659515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:57.659515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 00:34:57.659515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:34:57.663953 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 227/441] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Aug 26 00:34:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:34:57.664876 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb0789-5341-40e9-96b3-2ac43707970e None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:57.673739 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d60b6-689c-4084-a4a8-91e70bee51c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:57.674380 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d60b6-689c-4084-a4a8-91e70bee51c4 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:57.674669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d60b6-689c-4084-a4a8-91e70bee51c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:57.679103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d60b6-689c-4084-a4a8-91e70bee51c4 None admin] PATH_INFO: `/v3/projects/22414db73cef49549c4930bec9f6b5dc` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:57.679103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d60b6-689c-4084-a4a8-91e70bee51c4 None admin] RBAC: Authorizing `identity:get_project(project_id=22414db73cef49549c4930bec9f6b5dc)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:57.679103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d60b6-689c-4084-a4a8-91e70bee51c4 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:57.679103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 215/442] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:57 2022] GET /identity/v3/projects/22414db73cef49549c4930bec9f6b5dc => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:57.679854 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d60b6-689c-4084-a4a8-91e70bee51c4 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:57.776254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb0789-5341-40e9-96b3-2ac43707970e None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:57.784430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:57.793848 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d60b6-689c-4084-a4a8-91e70bee51c4 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:57.810094 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9bb56-a380-47d7-b6f2-6b1c3b2b7dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:57.810730 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9bb56-a380-47d7-b6f2-6b1c3b2b7dc7 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:57.811022 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9bb56-a380-47d7-b6f2-6b1c3b2b7dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:57.811284 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9bb56-a380-47d7-b6f2-6b1c3b2b7dc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:57.821444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e9bb56-a380-47d7-b6f2-6b1c3b2b7dc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:34:57.826640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-f2e9bb56-a380-47d7-b6f2-6b1c3b2b7dc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:34:57.826640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:34:57.826640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:34:57.826640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:34:57.826640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:34:57.826640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:34:57.826640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:34:57.826640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:34:57.826640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:34:57.826640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:34:57.826640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:34:57.826640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:34:57.826640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:34:57.826640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:34:57.826640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:34:57.826640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:34:57.827538 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:34:57.827538 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:34:57.827538 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:34:57.827538 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:34:57.827538 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:34:57.827538 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:34:57.827538 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:34:57.827538 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:34:57.827538 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:34:57.827538 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:34:57.827538 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:34:57.827538 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:34:57.827538 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:34:57.827538 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:34:57.827538 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:34:57.827538 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:34:57.828346 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:34:57.828346 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:34:57.828346 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:34:57.828346 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:34:57.828346 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:34:57.828346 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:34:57.828346 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:34:57.828346 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:34:57.828346 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:34:57.828346 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:34:57.828346 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:34:57.828346 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:34:57.830391 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 228/443] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:34:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:34:57.831394 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9bb56-a380-47d7-b6f2-6b1c3b2b7dc7 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:57.834983 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9a5fb-19c2-4096-9745-d52999e23b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:57.835715 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9a5fb-19c2-4096-9745-d52999e23b73 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:57.836035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9a5fb-19c2-4096-9745-d52999e23b73 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:57.836353 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9a5fb-19c2-4096-9745-d52999e23b73 None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:57.837173 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9a5fb-19c2-4096-9745-d52999e23b73 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:57.837520 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9a5fb-19c2-4096-9745-d52999e23b73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:57.839082 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9a5fb-19c2-4096-9745-d52999e23b73 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:57.848536 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 216/444] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:34:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:57.849353 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9a5fb-19c2-4096-9745-d52999e23b73 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:57.898182 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:57.943010 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9bb56-a380-47d7-b6f2-6b1c3b2b7dc7 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:57.947370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9a5fb-19c2-4096-9745-d52999e23b73 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:57.977603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e3846-b7ed-4c04-8750-db7dab4dc75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:57.978239 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e3846-b7ed-4c04-8750-db7dab4dc75e None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:57.978524 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e3846-b7ed-4c04-8750-db7dab4dc75e None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:57.978800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e3846-b7ed-4c04-8750-db7dab4dc75e None admin] PATH_INFO: `/v3/groups/008b22bc962749f88dcab89d95b610bd` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:57.979929 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e3846-b7ed-4c04-8750-db7dab4dc75e None admin] RBAC: Authorizing `identity:get_group(group_id=008b22bc962749f88dcab89d95b610bd)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:57.981154 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd5a0b-3b76-41ad-aaa4-cb6e1cc01b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:57.981499 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e3846-b7ed-4c04-8750-db7dab4dc75e None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:57.981891 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd5a0b-3b76-41ad-aaa4-cb6e1cc01b20 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:57.982233 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd5a0b-3b76-41ad-aaa4-cb6e1cc01b20 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:57.982567 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd5a0b-3b76-41ad-aaa4-cb6e1cc01b20 None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:57.983072 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 229/445] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Aug 26 00:34:57 2022] GET /identity/v3/groups/008b22bc962749f88dcab89d95b610bd => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:57.983297 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd5a0b-3b76-41ad-aaa4-cb6e1cc01b20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:57.983706 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd5a0b-3b76-41ad-aaa4-cb6e1cc01b20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:57.983837 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e3846-b7ed-4c04-8750-db7dab4dc75e None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:57.985327 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd5a0b-3b76-41ad-aaa4-cb6e1cc01b20 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:57.994129 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 217/446] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:34:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:57.994929 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd5a0b-3b76-41ad-aaa4-cb6e1cc01b20 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:58.019836 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:58.085048 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e3846-b7ed-4c04-8750-db7dab4dc75e None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:58.093522 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd5a0b-3b76-41ad-aaa4-cb6e1cc01b20 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:58.106735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:58.118899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a39b0-d920-4a3f-b685-c76adc15386b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:58.119507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a39b0-d920-4a3f-b685-c76adc15386b None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:58.119906 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a39b0-d920-4a3f-b685-c76adc15386b None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:58.120045 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a39b0-d920-4a3f-b685-c76adc15386b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:58.127086 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39957e-a6e4-4b97-9da0-8f9d49165090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:58.127684 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39957e-a6e4-4b97-9da0-8f9d49165090 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:58.127966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39957e-a6e4-4b97-9da0-8f9d49165090 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:58.128241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39957e-a6e4-4b97-9da0-8f9d49165090 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:58.128999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39957e-a6e4-4b97-9da0-8f9d49165090 None admin] RBAC: Adding query filter params (group.id=008b22bc962749f88dcab89d95b610bd, role.id=5032636d8729461c9926a59b9bcd041b, scope.project.id=22414db73cef49549c4930bec9f6b5dc) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:58.129308 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39957e-a6e4-4b97-9da0-8f9d49165090 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:58.130735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39957e-a6e4-4b97-9da0-8f9d49165090 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:58.136221 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 218/447] 10.222.0.59 () {56 vars in 1376 bytes} [Fri Aug 26 00:34:57 2022] GET /identity/v3/role_assignments?group.id=008b22bc962749f88dcab89d95b610bd&scope.project.id=22414db73cef49549c4930bec9f6b5dc&role.id=5032636d8729461c9926a59b9bcd041b => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:58.137041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39957e-a6e4-4b97-9da0-8f9d49165090 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:58.193476 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a39b0-d920-4a3f-b685-c76adc15386b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:58.195091 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a39b0-d920-4a3f-b685-c76adc15386b None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:58.231522 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:58.266807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39957e-a6e4-4b97-9da0-8f9d49165090 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:58.310107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd5db8-1404-43e4-900a-93132a077832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:58.310727 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd5db8-1404-43e4-900a-93132a077832 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:58.311014 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd5db8-1404-43e4-900a-93132a077832 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:58.311290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd5db8-1404-43e4-900a-93132a077832 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:58.321398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 230/448] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:34:58.327236 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a39b0-d920-4a3f-b685-c76adc15386b None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:58.362319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:58.388977 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd5db8-1404-43e4-900a-93132a077832 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:58.390496 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd5db8-1404-43e4-900a-93132a077832 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:58.448970 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a39b0-d920-4a3f-b685-c76adc15386b None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:58.488697 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028a823-ea50-4260-8a1d-10ac7898bcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:58.489172 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028a823-ea50-4260-8a1d-10ac7898bcbd None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:58.489450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028a823-ea50-4260-8a1d-10ac7898bcbd None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:58.489720 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028a823-ea50-4260-8a1d-10ac7898bcbd None admin] PATH_INFO: `/v3/projects/22414db73cef49549c4930bec9f6b5dc` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:58.491731 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c028a823-ea50-4260-8a1d-10ac7898bcbd None admin] RBAC: Authorizing `identity:get_project(project_id=22414db73cef49549c4930bec9f6b5dc)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:58.493277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c028a823-ea50-4260-8a1d-10ac7898bcbd None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:58.497283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 231/449] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:58 2022] GET /identity/v3/projects/22414db73cef49549c4930bec9f6b5dc => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:58.500380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028a823-ea50-4260-8a1d-10ac7898bcbd None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:58.503956 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:58.508539 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 219/450] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:34:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:34:58.514232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd5db8-1404-43e4-900a-93132a077832 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:58.591588 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:58.603976 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028a823-ea50-4260-8a1d-10ac7898bcbd None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:58.613555 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd5db8-1404-43e4-900a-93132a077832 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:58.636351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014865f1-0ffb-47d6-9b27-25c55aa55ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:58.637010 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014865f1-0ffb-47d6-9b27-25c55aa55ce9 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:58.637292 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014865f1-0ffb-47d6-9b27-25c55aa55ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:58.637566 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014865f1-0ffb-47d6-9b27-25c55aa55ce9 None admin] PATH_INFO: `/v3/groups/008b22bc962749f88dcab89d95b610bd` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:58.638688 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014865f1-0ffb-47d6-9b27-25c55aa55ce9 None admin] RBAC: Authorizing `identity:get_group(group_id=008b22bc962749f88dcab89d95b610bd)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:58.640126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014865f1-0ffb-47d6-9b27-25c55aa55ce9 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:58.642228 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 232/451] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Aug 26 00:34:58 2022] GET /identity/v3/groups/008b22bc962749f88dcab89d95b610bd => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:58.647046 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df3770-f10c-46b4-b528-1347379e7df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:58.647677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df3770-f10c-46b4-b528-1347379e7df2 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:58.647933 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014865f1-0ffb-47d6-9b27-25c55aa55ce9 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:58.648036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df3770-f10c-46b4-b528-1347379e7df2 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:58.648218 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df3770-f10c-46b4-b528-1347379e7df2 None admin] PATH_INFO: `/v3/projects/86ade54a72a44599b7a6313f909a3551` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:58.649377 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df3770-f10c-46b4-b528-1347379e7df2 None admin] RBAC: Authorizing `identity:get_project(project_id=86ade54a72a44599b7a6313f909a3551)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:58.650966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df3770-f10c-46b4-b528-1347379e7df2 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:58.657306 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 220/452] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:34:58 2022] GET /identity/v3/projects/86ade54a72a44599b7a6313f909a3551 => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:58.658824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df3770-f10c-46b4-b528-1347379e7df2 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:58.715870 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:58.745599 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014865f1-0ffb-47d6-9b27-25c55aa55ce9 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:58.754712 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df3770-f10c-46b4-b528-1347379e7df2 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:58.778968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dda7a6-eaff-465d-9921-8315a712507a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:58.779557 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dda7a6-eaff-465d-9921-8315a712507a None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:58.779834 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dda7a6-eaff-465d-9921-8315a712507a None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:58.780105 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dda7a6-eaff-465d-9921-8315a712507a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:58.780853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dda7a6-eaff-465d-9921-8315a712507a None admin] RBAC: Adding query filter params (group.id=008b22bc962749f88dcab89d95b610bd, role.id=423aa6ac8b05402da7587e83e5c4ce09, scope.project.id=22414db73cef49549c4930bec9f6b5dc) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:58.781160 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dda7a6-eaff-465d-9921-8315a712507a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:58.783109 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dda7a6-eaff-465d-9921-8315a712507a None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:58.788960 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049508b2-356b-4a78-b65b-b6b546d28a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:58.789432 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 233/453] 10.222.0.59 () {56 vars in 1376 bytes} [Fri Aug 26 00:34:58 2022] GET /identity/v3/role_assignments?group.id=008b22bc962749f88dcab89d95b610bd&scope.project.id=22414db73cef49549c4930bec9f6b5dc&role.id=423aa6ac8b05402da7587e83e5c4ce09 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:34:58.789595 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049508b2-356b-4a78-b65b-b6b546d28a74 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:58.789878 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049508b2-356b-4a78-b65b-b6b546d28a74 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:58.790153 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049508b2-356b-4a78-b65b-b6b546d28a74 None admin] PATH_INFO: `/v3/groups/bf671cb739444d1798469f5fc7dfaa16` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:58.791312 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049508b2-356b-4a78-b65b-b6b546d28a74 None admin] RBAC: Authorizing `identity:get_group(group_id=bf671cb739444d1798469f5fc7dfaa16)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:58.792783 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049508b2-356b-4a78-b65b-b6b546d28a74 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:58.794395 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 221/454] 10.222.0.59 () {56 vars in 1180 bytes} [Fri Aug 26 00:34:58 2022] GET /identity/v3/groups/bf671cb739444d1798469f5fc7dfaa16 => generated 229 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:34:58.800666 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dda7a6-eaff-465d-9921-8315a712507a None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:34:58.904771 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dda7a6-eaff-465d-9921-8315a712507a None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:34:58.940417 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636fb6e-9eb9-4005-9525-9caa6f0b782f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:34:58.941051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636fb6e-9eb9-4005-9525-9caa6f0b782f None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:34:58.941327 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636fb6e-9eb9-4005-9525-9caa6f0b782f None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:34:58.941597 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636fb6e-9eb9-4005-9525-9caa6f0b782f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:34:58.942319 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636fb6e-9eb9-4005-9525-9caa6f0b782f None admin] RBAC: Adding query filter params (group.id=bf671cb739444d1798469f5fc7dfaa16, role.id=1a33e3c8cb8347feb9498595886cc621, scope.project.id=86ade54a72a44599b7a6313f909a3551) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:34:58.942621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636fb6e-9eb9-4005-9525-9caa6f0b782f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:34:58.944040 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636fb6e-9eb9-4005-9525-9caa6f0b782f None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:34:58.945803 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:34:58.950548 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 234/455] 10.222.0.59 () {56 vars in 1376 bytes} [Fri Aug 26 00:34:58 2022] GET /identity/v3/role_assignments?group.id=bf671cb739444d1798469f5fc7dfaa16&scope.project.id=86ade54a72a44599b7a6313f909a3551&role.id=1a33e3c8cb8347feb9498595886cc621 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:01.252302 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45353548-5443-491a-b80d-1e14d282af34 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:01.253182 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45353548-5443-491a-b80d-1e14d282af34 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:01.253621 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45353548-5443-491a-b80d-1e14d282af34 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:01.255458 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 222/456] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:01.260526 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a3324-1139-4c1e-b80b-9d0049a71304 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:01.261348 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a3324-1139-4c1e-b80b-9d0049a71304 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:01.261774 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a3324-1139-4c1e-b80b-9d0049a71304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:01.283207 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-4f9a3324-1139-4c1e-b80b-9d0049a71304 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:01.284627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-4f9a3324-1139-4c1e-b80b-9d0049a71304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:01.372619 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 235/457] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:35:01.588773 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d5e1b-8d26-479b-bfc3-eeabcafbd970 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:01.589059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d5e1b-8d26-479b-bfc3-eeabcafbd970 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:01.589336 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d5e1b-8d26-479b-bfc3-eeabcafbd970 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:01.590895 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 223/458] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:01.597043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e75fc1-0965-4894-a8b6-0a892ed3b3a4 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:01.597338 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e75fc1-0965-4894-a8b6-0a892ed3b3a4 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:01.597622 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e75fc1-0965-4894-a8b6-0a892ed3b3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:01.625532 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-a2e75fc1-0965-4894-a8b6-0a892ed3b3a4 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:01.626734 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a2e75fc1-0965-4894-a8b6-0a892ed3b3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:01.715049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 236/459] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:35:01.818958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d5e1b-8d26-479b-bfc3-eeabcafbd970 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:01.835171 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:01.872702 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80526f43-5a8c-4c62-8545-e498dbbdb3e8 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:01.872979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80526f43-5a8c-4c62-8545-e498dbbdb3e8 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:01.873182 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80526f43-5a8c-4c62-8545-e498dbbdb3e8 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:01.874748 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 237/460] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:01.880521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60decf7-7744-46ab-9760-59dc783dec88 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:01.880814 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60decf7-7744-46ab-9760-59dc783dec88 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:01.881093 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60decf7-7744-46ab-9760-59dc783dec88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:01.913018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-f60decf7-7744-46ab-9760-59dc783dec88 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:01.914212 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f60decf7-7744-46ab-9760-59dc783dec88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:01.977466 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d5e1b-8d26-479b-bfc3-eeabcafbd970 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:02.001973 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:02.019021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf699b83-363b-47a1-8648-5e4455fd8614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:02.019951 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf699b83-363b-47a1-8648-5e4455fd8614 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:02.020411 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf699b83-363b-47a1-8648-5e4455fd8614 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:02.020852 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf699b83-363b-47a1-8648-5e4455fd8614 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:02.021573 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 238/461] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:35:02.023330 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad2b1b-9881-4d62-9304-47452994bfe8 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:02.023706 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad2b1b-9881-4d62-9304-47452994bfe8 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:02.024563 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad2b1b-9881-4d62-9304-47452994bfe8 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:02.031035 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 239/462] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:02.034236 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a21139-4de8-4336-a0d8-c9bb86adbd9e None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:02.034631 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a21139-4de8-4336-a0d8-c9bb86adbd9e None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:02.035298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a21139-4de8-4336-a0d8-c9bb86adbd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:02.036054 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf699b83-363b-47a1-8648-5e4455fd8614 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:02.037758 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf699b83-363b-47a1-8648-5e4455fd8614 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:02.055928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-e1a21139-4de8-4336-a0d8-c9bb86adbd9e None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:02.056098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-bf699b83-363b-47a1-8648-5e4455fd8614 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:02.056098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:02.056098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:02.056098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:02.056098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:02.056098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:02.056098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:02.056098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:02.056098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:02.056098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:02.056098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:02.056098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:02.056098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:02.056098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:02.056098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:02.056098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:02.057140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:02.057140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:02.057140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:02.057140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:02.057140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:02.057140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:02.057140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:02.057140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:02.057140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:02.057140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:02.057140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:02.057140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:02.057140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:02.057140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:02.057140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:02.057140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:02.057140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:02.057987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:02.057987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:02.057987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:02.058167 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 224/463] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:02.059817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-e1a21139-4de8-4336-a0d8-c9bb86adbd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:02.062910 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf699b83-363b-47a1-8648-5e4455fd8614 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:02.126001 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:02.131589 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 240/464] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:35:02.132342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a21139-4de8-4336-a0d8-c9bb86adbd9e None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:02.166846 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf699b83-363b-47a1-8648-5e4455fd8614 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:02.201654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9075ec-b0b6-4ac0-8046-f674336012e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:02.202603 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9075ec-b0b6-4ac0-8046-f674336012e4 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:02.203093 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9075ec-b0b6-4ac0-8046-f674336012e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:02.203624 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9075ec-b0b6-4ac0-8046-f674336012e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:02.214778 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9075ec-b0b6-4ac0-8046-f674336012e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:02.216466 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9075ec-b0b6-4ac0-8046-f674336012e4 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:02.231526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-0f9075ec-b0b6-4ac0-8046-f674336012e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:02.231526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:02.231526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:02.231526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:02.231526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:02.231526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:02.231526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:02.231526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:02.231526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:02.231526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:02.231526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:02.231526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:02.231526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:02.231526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:02.231526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:02.231526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:02.232415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:02.232415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:02.232415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:02.232415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:02.232415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:02.232415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:02.232415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:02.232415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:02.232415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:02.232415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:02.232415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:02.232415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:02.232415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:02.232415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:02.232415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:02.232415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:02.232415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:02.233429 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:02.233429 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:02.233429 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:02.235489 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 225/465] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:02.241251 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a21139-4de8-4336-a0d8-c9bb86adbd9e None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:02.242256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9075ec-b0b6-4ac0-8046-f674336012e4 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:02.259933 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:02.277816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd8c8e-9543-422f-b4e7-9f4ec432dd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:02.281006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd8c8e-9543-422f-b4e7-9f4ec432dd0e None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:02.281006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd8c8e-9543-422f-b4e7-9f4ec432dd0e None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:02.281006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd8c8e-9543-422f-b4e7-9f4ec432dd0e None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:02.281006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd8c8e-9543-422f-b4e7-9f4ec432dd0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:02.281496 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd8c8e-9543-422f-b4e7-9f4ec432dd0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:02.283056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd8c8e-9543-422f-b4e7-9f4ec432dd0e None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:02.295217 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 241/466] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:02.301131 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd8c8e-9543-422f-b4e7-9f4ec432dd0e None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:02.428982 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd8c8e-9543-422f-b4e7-9f4ec432dd0e None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:02.435133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9075ec-b0b6-4ac0-8046-f674336012e4 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:02.455506 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:02.475419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d5ce0-7075-481b-9be3-e67a915a616a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:02.476372 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d5ce0-7075-481b-9be3-e67a915a616a None admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:02.476928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d5ce0-7075-481b-9be3-e67a915a616a None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:02.477153 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4ca79a-6c86-4b18-8d3e-d85138a2db2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:02.477518 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d5ce0-7075-481b-9be3-e67a915a616a None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:02.477951 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ca79a-6c86-4b18-8d3e-d85138a2db2d None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:02.478337 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ca79a-6c86-4b18-8d3e-d85138a2db2d None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:02.478709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ca79a-6c86-4b18-8d3e-d85138a2db2d None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:02.478929 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449d5ce0-7075-481b-9be3-e67a915a616a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:02.479558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4ca79a-6c86-4b18-8d3e-d85138a2db2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:02.479966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4ca79a-6c86-4b18-8d3e-d85138a2db2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:02.480873 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449d5ce0-7075-481b-9be3-e67a915a616a None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:02.481621 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4ca79a-6c86-4b18-8d3e-d85138a2db2d None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:02.494811 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 226/467] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:02.495644 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4ca79a-6c86-4b18-8d3e-d85138a2db2d None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:02.596515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 242/468] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Aug 26 00:35:02 2022] POST /identity/v3/users => generated 246 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:35:02.596515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d5ce0-7075-481b-9be3-e67a915a616a None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:02.613182 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4ca79a-6c86-4b18-8d3e-d85138a2db2d None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:02.636549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:02.664767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a181fff-3ff2-4953-99f8-432f7b0c6c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:02.665600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a181fff-3ff2-4953-99f8-432f7b0c6c6c None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:02.666049 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a181fff-3ff2-4953-99f8-432f7b0c6c6c None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:02.666714 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a181fff-3ff2-4953-99f8-432f7b0c6c6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:02.686777 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a181fff-3ff2-4953-99f8-432f7b0c6c6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:02.686777 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a181fff-3ff2-4953-99f8-432f7b0c6c6c None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:02.703511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-5a181fff-3ff2-4953-99f8-432f7b0c6c6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:02.703511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:02.703511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:02.703511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:02.703511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:02.703511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:02.703511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:02.703511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:02.703511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:02.703511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:02.703511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:02.703511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:02.703511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:02.703511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:02.703511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:02.703511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:02.704584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:02.704584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:02.704584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:02.704584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:02.704584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:02.704584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:02.704584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:02.704584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:02.704584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:02.704584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:02.704584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:02.704584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:02.704584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:02.704584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:02.704584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:02.704584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:02.704584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:02.709585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:02.709585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:02.709585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:02.709585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 227/469] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:02.709585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a181fff-3ff2-4953-99f8-432f7b0c6c6c None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:02.723133 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d5ce0-7075-481b-9be3-e67a915a616a None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:02.747610 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:02.762433 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7411dda-e736-4c1a-81d5-7c4686bd6d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:02.763368 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7411dda-e736-4c1a-81d5-7c4686bd6d86 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:02.763808 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7411dda-e736-4c1a-81d5-7c4686bd6d86 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:02.764238 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7411dda-e736-4c1a-81d5-7c4686bd6d86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:02.777408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7411dda-e736-4c1a-81d5-7c4686bd6d86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:02.779917 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7411dda-e736-4c1a-81d5-7c4686bd6d86 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:02.796383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-f7411dda-e736-4c1a-81d5-7c4686bd6d86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:02.796383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:02.796383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:02.796383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:02.796383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:02.796383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:02.796383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:02.796383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:02.796383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:02.796383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:02.796383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:02.796383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:02.796383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:02.796383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:02.796383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:02.796383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:02.798000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:02.798000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:02.798000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:02.798000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:02.798000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:02.798000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:02.798000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:02.798000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:02.798000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:02.798000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:02.798000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:02.798000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:02.798000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:02.798000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:02.798000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:02.798000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:02.798000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:02.798849 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:02.798849 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:02.798849 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:02.799727 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 243/470] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:02.800128 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7411dda-e736-4c1a-81d5-7c4686bd6d86 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:02.847998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a181fff-3ff2-4953-99f8-432f7b0c6c6c None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:02.886515 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897b5ee-6961-4b07-9e8d-a81954a91bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:02.887236 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897b5ee-6961-4b07-9e8d-a81954a91bf4 None admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:02.887519 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897b5ee-6961-4b07-9e8d-a81954a91bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:02.887793 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897b5ee-6961-4b07-9e8d-a81954a91bf4 None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:02.888446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5897b5ee-6961-4b07-9e8d-a81954a91bf4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:02.889925 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5897b5ee-6961-4b07-9e8d-a81954a91bf4 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:02.903767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:02.919454 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7411dda-e736-4c1a-81d5-7c4686bd6d86 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:02.953057 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc083ce-625c-4e90-9275-945faae92acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:02.954140 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc083ce-625c-4e90-9275-945faae92acf None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:02.954418 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc083ce-625c-4e90-9275-945faae92acf None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:02.954738 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc083ce-625c-4e90-9275-945faae92acf None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:02.955391 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc083ce-625c-4e90-9275-945faae92acf None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:02.955691 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc083ce-625c-4e90-9275-945faae92acf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:02.957175 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc083ce-625c-4e90-9275-945faae92acf None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:02.972510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 228/471] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Aug 26 00:35:02 2022] POST /identity/v3/users => generated 248 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:35:02.972510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897b5ee-6961-4b07-9e8d-a81954a91bf4 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:02.972782 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 244/472] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:02.972782 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc941ec4-ed3d-4a7e-8086-8dd74be63cf6 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:02.972782 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc941ec4-ed3d-4a7e-8086-8dd74be63cf6 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:02.972782 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc941ec4-ed3d-4a7e-8086-8dd74be63cf6 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:02.973517 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 245/473] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:02.980193 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6debc-89a7-4aa4-b968-7cfb5c17cb44 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:02.980505 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6debc-89a7-4aa4-b968-7cfb5c17cb44 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:02.980779 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6debc-89a7-4aa4-b968-7cfb5c17cb44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:03.002243 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-bae6debc-89a7-4aa4-b968-7cfb5c17cb44 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:03.003498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-bae6debc-89a7-4aa4-b968-7cfb5c17cb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:03.092092 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:03.097564 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 246/474] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:35:03.106207 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6debc-89a7-4aa4-b968-7cfb5c17cb44 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:03.110617 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897b5ee-6961-4b07-9e8d-a81954a91bf4 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:03.156524 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafd879-b650-4b3d-b3e2-aea84c16c60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:03.157889 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafd879-b650-4b3d-b3e2-aea84c16c60c None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:03.158173 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafd879-b650-4b3d-b3e2-aea84c16c60c None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:03.158486 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafd879-b650-4b3d-b3e2-aea84c16c60c None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:03.159148 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafd879-b650-4b3d-b3e2-aea84c16c60c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:03.159448 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafd879-b650-4b3d-b3e2-aea84c16c60c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:03.160832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafd879-b650-4b3d-b3e2-aea84c16c60c None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:03.173971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 229/475] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:03.179883 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafd879-b650-4b3d-b3e2-aea84c16c60c None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:03.222902 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:03.246403 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6debc-89a7-4aa4-b968-7cfb5c17cb44 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:03.286207 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb40209-a0a0-424f-a658-2c9e31874ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:03.286833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb40209-a0a0-424f-a658-2c9e31874ab3 None admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:03.287101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb40209-a0a0-424f-a658-2c9e31874ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:03.287377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb40209-a0a0-424f-a658-2c9e31874ab3 None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:03.288027 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb40209-a0a0-424f-a658-2c9e31874ab3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:03.289522 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb40209-a0a0-424f-a658-2c9e31874ab3 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:03.315746 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafd879-b650-4b3d-b3e2-aea84c16c60c None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:03.354385 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f3a27-c5fe-4a96-918b-b438faad50e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:03.354996 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f3a27-c5fe-4a96-918b-b438faad50e0 None admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:03.355275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f3a27-c5fe-4a96-918b-b438faad50e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:03.355549 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f3a27-c5fe-4a96-918b-b438faad50e0 None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:03.356326 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060f3a27-c5fe-4a96-918b-b438faad50e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:03.357900 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060f3a27-c5fe-4a96-918b-b438faad50e0 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:03.376588 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 247/476] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Aug 26 00:35:03 2022] POST /identity/v3/users => generated 248 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:35:03.442648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb40209-a0a0-424f-a658-2c9e31874ab3 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:03.451135 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:03.458891 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 230/477] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Aug 26 00:35:03 2022] POST /identity/v3/users => generated 249 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:35:03.560255 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb40209-a0a0-424f-a658-2c9e31874ab3 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:03.593579 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:03.598540 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c82d6-cfa7-4567-9cf0-2065adca05bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:03.599174 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c82d6-cfa7-4567-9cf0-2065adca05bf None admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:03.599712 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c82d6-cfa7-4567-9cf0-2065adca05bf None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:03.599977 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c82d6-cfa7-4567-9cf0-2065adca05bf None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:03.600702 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c82d6-cfa7-4567-9cf0-2065adca05bf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:03.602109 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c82d6-cfa7-4567-9cf0-2065adca05bf None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:03.649285 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.sql.core [None req-cc1c82d6-cfa7-4567-9cf0-2065adca05bf None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 26 00:35:03.649285 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 26 00:35:03.649285 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [parameters: {'id': 'f0bf38cd9b354dbaaa7f347a1de30a3e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 26 00:35:03.649285 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88254) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 26 00:35:03.674088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-cc1c82d6-cfa7-4567-9cf0-2065adca05bf None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 26 00:35:03.674088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:03.674088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 00:35:03.674088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 00:35:03.674088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 00:35:03.674088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 00:35:03.674088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 00:35:03.674088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 00:35:03.674088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 00:35:03.674088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application conn.query(q) Aug 26 00:35:03.674088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 00:35:03.674088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 00:35:03.674088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 00:35:03.674088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application result.read() Aug 26 00:35:03.674088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 00:35:03.675023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 00:35:03.675023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 00:35:03.675023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 00:35:03.675023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 00:35:03.675023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 00:35:03.675023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 00:35:03.675023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 00:35:03.675023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 26 00:35:03.675023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:03.675023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 26 00:35:03.675023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:03.675023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:03.675023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 26 00:35:03.675023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 26 00:35:03.675023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 26 00:35:03.675023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return ref.to_dict() Aug 26 00:35:03.675023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 00:35:03.675023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application next(self.gen) Aug 26 00:35:03.675948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 26 00:35:03.675948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application yield resource Aug 26 00:35:03.675948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 00:35:03.675948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application next(self.gen) Aug 26 00:35:03.675948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 26 00:35:03.675948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self.session.rollback() Aug 26 00:35:03.675948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 26 00:35:03.675948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self.force_reraise() Aug 26 00:35:03.675948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 26 00:35:03.675948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise self.value Aug 26 00:35:03.675948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 26 00:35:03.675948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 26 00:35:03.675948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 26 00:35:03.675948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application session.commit() Aug 26 00:35:03.675948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 26 00:35:03.675948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 26 00:35:03.675948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 26 00:35:03.677380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._prepare_impl() Aug 26 00:35:03.677380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 26 00:35:03.677380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self.session.flush() Aug 26 00:35:03.677380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 26 00:35:03.677380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._flush(objects) Aug 26 00:35:03.677380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 26 00:35:03.677380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 26 00:35:03.677380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 26 00:35:03.677380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application compat.raise_( Aug 26 00:35:03.677380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 00:35:03.677380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception Aug 26 00:35:03.677380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 26 00:35:03.677380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application flush_context.execute() Aug 26 00:35:03.677380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 26 00:35:03.677380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rec.execute(self) Aug 26 00:35:03.677380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 26 00:35:03.677380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 26 00:35:03.678253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 26 00:35:03.678253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 26 00:35:03.678253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 26 00:35:03.678253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 26 00:35:03.678253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 26 00:35:03.678253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 26 00:35:03.678253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 26 00:35:03.678253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 26 00:35:03.678253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 26 00:35:03.678253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 26 00:35:03.678253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 26 00:35:03.678253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 26 00:35:03.678253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 26 00:35:03.678253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 26 00:35:03.678253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 00:35:03.678253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception Aug 26 00:35:03.679076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 00:35:03.679076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 00:35:03.679076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 00:35:03.679076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 00:35:03.679076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 00:35:03.679076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 00:35:03.679076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 00:35:03.679076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application conn.query(q) Aug 26 00:35:03.679076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 00:35:03.679076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 00:35:03.679076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 00:35:03.679076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application result.read() Aug 26 00:35:03.679076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 00:35:03.679076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 00:35:03.679076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 00:35:03.679076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 00:35:03.681387 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 00:35:03.681387 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 00:35:03.681387 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 00:35:03.681387 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 00:35:03.681387 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 26 00:35:03.681387 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 26 00:35:03.681387 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [parameters: {'id': 'f0bf38cd9b354dbaaa7f347a1de30a3e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 26 00:35:03.681387 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 26 00:35:03.681387 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:03.681387 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:35:03.681387 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:03.681387 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:03.681387 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:03.681387 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:03.681387 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:03.681387 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:03.683305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:03.683305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:03.683305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:03.683305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:03.683305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:03.683305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:03.683305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 26 00:35:03.683305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 26 00:35:03.683305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:03.683305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:03.683305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 26 00:35:03.683305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 26 00:35:03.683305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 26 00:35:03.683305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 26 00:35:03.683305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 26 00:35:03.683305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:03.694367 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 248/478] 10.222.0.59 () {60 vars in 1131 bytes} [Fri Aug 26 00:35:03 2022] POST /identity/v3/roles => generated 144 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:03.694559 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f3a27-c5fe-4a96-918b-b438faad50e0 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:03.793957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f3a27-c5fe-4a96-918b-b438faad50e0 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:03.835811 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fc569-6c8f-456f-b5fa-8acdab4dac2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:03.836370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fc569-6c8f-456f-b5fa-8acdab4dac2d None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:03.836679 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fc569-6c8f-456f-b5fa-8acdab4dac2d None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:03.836953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fc569-6c8f-456f-b5fa-8acdab4dac2d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:03.849893 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fc569-6c8f-456f-b5fa-8acdab4dac2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:03.855234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-292fc569-6c8f-456f-b5fa-8acdab4dac2d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 00:35:03.855234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:03.855234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:03.855234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:03.855234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:03.855234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:03.855234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:03.855234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:03.855234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:03.855234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:03.855234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:03.855234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:03.855234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:03.855234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:03.855234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:03.855234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:03.856152 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:03.856152 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:03.856152 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:03.856152 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:03.856152 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:03.856152 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:03.856152 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:03.856152 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:03.856152 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:03.856152 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:03.856152 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:03.856152 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:03.856152 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:03.856152 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:03.856152 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:03.856152 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:03.857013 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:03.857013 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:03.857013 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:03.857013 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:03.857013 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:03.857013 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:03.857013 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:03.857013 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:03.857013 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:03.857013 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:03.857013 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 00:35:03.857013 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:03.857013 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 231/479] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Aug 26 00:35:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:03.862793 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c82d6-cfa7-4567-9cf0-2065adca05bf None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:03.971906 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c82d6-cfa7-4567-9cf0-2065adca05bf None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:04.005836 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32728ba4-1f30-4a02-9493-62996efbe762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:04.006450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32728ba4-1f30-4a02-9493-62996efbe762 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:04.006732 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32728ba4-1f30-4a02-9493-62996efbe762 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:04.013750 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32728ba4-1f30-4a02-9493-62996efbe762 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:04.013750 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32728ba4-1f30-4a02-9493-62996efbe762 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:04.013750 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32728ba4-1f30-4a02-9493-62996efbe762 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:04.013750 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32728ba4-1f30-4a02-9493-62996efbe762 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:04.018486 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 249/480] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:04.997703 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca365ca-719d-4367-ada4-7b2fb85bc2d6 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:04.998719 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca365ca-719d-4367-ada4-7b2fb85bc2d6 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:04.999207 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca365ca-719d-4367-ada4-7b2fb85bc2d6 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:05.001584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 232/481] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:05.007486 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcf0da-f549-4050-a985-60f3f4822d7b None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:05.007946 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcf0da-f549-4050-a985-60f3f4822d7b None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:05.008023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcf0da-f549-4050-a985-60f3f4822d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:05.032762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-7adcf0da-f549-4050-a985-60f3f4822d7b None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:05.033934 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-7adcf0da-f549-4050-a985-60f3f4822d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:05.107964 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:05.128441 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 250/482] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:35:05.548723 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1199a42-ffa4-4d43-bddc-d699ad60302c None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:05.549049 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1199a42-ffa4-4d43-bddc-d699ad60302c None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:05.549271 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1199a42-ffa4-4d43-bddc-d699ad60302c None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:05.550856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 233/483] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:05.552739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcf0da-f549-4050-a985-60f3f4822d7b None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:05.556046 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54db54-0594-467f-94bb-6c2a311281d7 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:05.556328 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54db54-0594-467f-94bb-6c2a311281d7 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:05.556627 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54db54-0594-467f-94bb-6c2a311281d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:05.586291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-8b54db54-0594-467f-94bb-6c2a311281d7 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:05.587518 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-8b54db54-0594-467f-94bb-6c2a311281d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:05.673236 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:05.690642 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 234/484] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:35:05.691909 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8d74c-d648-45b2-9c16-1d0e376b3d97 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:05.692197 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8d74c-d648-45b2-9c16-1d0e376b3d97 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:05.692486 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8d74c-d648-45b2-9c16-1d0e376b3d97 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:05.694138 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 235/485] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:05.699625 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd608cb-1bf0-47a4-9db7-7304273688ff None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:05.699910 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd608cb-1bf0-47a4-9db7-7304273688ff None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:05.700182 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd608cb-1bf0-47a4-9db7-7304273688ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:05.714386 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcf0da-f549-4050-a985-60f3f4822d7b None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:05.729481 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-7bd608cb-1bf0-47a4-9db7-7304273688ff None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:05.730671 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-7bd608cb-1bf0-47a4-9db7-7304273688ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:05.751079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df1f23-dbc5-4c82-92ea-045d314a5021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:05.752151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df1f23-dbc5-4c82-92ea-045d314a5021 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:05.752542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df1f23-dbc5-4c82-92ea-045d314a5021 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:05.753096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df1f23-dbc5-4c82-92ea-045d314a5021 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:05.764347 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df1f23-dbc5-4c82-92ea-045d314a5021 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:05.769708 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-98df1f23-dbc5-4c82-92ea-045d314a5021 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:05.769708 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:05.769708 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:05.769708 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:05.769708 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:05.769708 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:05.769708 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:05.769708 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:05.769708 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:05.769708 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:05.769708 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:05.769708 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:05.769708 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:05.769708 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:05.769708 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:05.769708 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:05.770612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:05.770612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:05.770612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:05.770612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:05.770612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:05.770612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:05.770612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:05.770612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:05.770612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:05.770612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:05.770612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:05.770612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:05.770612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:05.770612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:05.770612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:05.770612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:05.771950 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:05.771950 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:05.771950 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:05.771950 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:05.771950 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:05.771950 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:05.771950 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:05.771950 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:05.771950 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:05.771950 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:05.771950 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:05.771950 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:05.771950 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 251/486] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Aug 26 00:35:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:05.776772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df1f23-dbc5-4c82-92ea-045d314a5021 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:05.834212 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:05.896775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 236/487] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:35:05.924821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df1f23-dbc5-4c82-92ea-045d314a5021 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:05.966537 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06819c-9656-4279-843a-2b6b60862930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:05.967071 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06819c-9656-4279-843a-2b6b60862930 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:05.967351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06819c-9656-4279-843a-2b6b60862930 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:05.967624 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06819c-9656-4279-843a-2b6b60862930 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:05.968293 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db06819c-9656-4279-843a-2b6b60862930 None admin] RBAC: Adding query filter params (name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:05.968624 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db06819c-9656-4279-843a-2b6b60862930 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:05.970037 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db06819c-9656-4279-843a-2b6b60862930 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:05.978529 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 252/488] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Aug 26 00:35:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:35:05.986076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd608cb-1bf0-47a4-9db7-7304273688ff None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:06.019794 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06819c-9656-4279-843a-2b6b60862930 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:06.162306 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06819c-9656-4279-843a-2b6b60862930 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:06.165285 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd608cb-1bf0-47a4-9db7-7304273688ff None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:06.173539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:06.188969 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:06.197779 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d1b42-cc93-4c37-a6a1-770e37471479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:06.198464 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d1b42-cc93-4c37-a6a1-770e37471479 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:06.198749 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d1b42-cc93-4c37-a6a1-770e37471479 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:06.199052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d1b42-cc93-4c37-a6a1-770e37471479 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:06.206504 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65318be2-93e3-47da-8366-d5b1b57294f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:06.208064 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65318be2-93e3-47da-8366-d5b1b57294f9 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:06.208597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65318be2-93e3-47da-8366-d5b1b57294f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:06.209122 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65318be2-93e3-47da-8366-d5b1b57294f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:06.210759 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99d1b42-cc93-4c37-a6a1-770e37471479 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:06.216056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-e99d1b42-cc93-4c37-a6a1-770e37471479 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:06.216056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:06.216056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:06.216056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:06.216056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:06.216056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:06.216056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:06.216056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:06.216056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:06.216056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:06.216056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:06.216056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:06.216056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:06.216056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:06.216056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:06.216056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:06.228665 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:06.228665 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:06.228665 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:06.228665 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:06.228665 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:06.228665 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:06.228665 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:06.228665 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:06.228665 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:06.228665 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:06.228665 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:06.228665 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:06.228665 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:06.228665 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:06.228665 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:06.228665 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:06.229498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:06.229498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:06.229498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:06.229498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:06.229498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:06.229498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:06.229498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:06.229498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:06.229498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:06.229498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:06.229498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:06.229498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:06.229498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 253/489] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Aug 26 00:35:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:06.229498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed5a23-e6d9-4d48-9f6d-ffc25b9169a1 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:06.230220 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed5a23-e6d9-4d48-9f6d-ffc25b9169a1 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:06.230220 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed5a23-e6d9-4d48-9f6d-ffc25b9169a1 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:06.230220 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 254/490] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:06.230220 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed5a23-e6d9-4d48-9f6d-ffc25b9169a1 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:06.237653 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65318be2-93e3-47da-8366-d5b1b57294f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:06.239783 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65318be2-93e3-47da-8366-d5b1b57294f9 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:06.258649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-65318be2-93e3-47da-8366-d5b1b57294f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:06.258649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:06.258649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:06.258649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:06.258649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:06.258649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:06.258649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:06.258649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:06.258649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:06.258649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:06.258649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:06.258649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:06.258649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:06.258649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:06.258649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:06.258649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:06.259816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:06.259816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:06.259816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:06.259816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:06.259816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:06.259816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:06.259816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:06.259816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:06.259816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:06.259816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:06.259816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:06.259816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:06.259816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:06.259816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:06.259816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:06.259816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:06.259816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:06.260788 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:06.260788 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:06.260788 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:06.262694 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 237/491] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:06.266506 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0025cec-ace0-405c-8529-c121dc9510a0 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:06.266781 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0025cec-ace0-405c-8529-c121dc9510a0 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:06.267049 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0025cec-ace0-405c-8529-c121dc9510a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:06.285193 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-a0025cec-ace0-405c-8529-c121dc9510a0 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:06.286122 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-a0025cec-ace0-405c-8529-c121dc9510a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:06.344153 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed5a23-e6d9-4d48-9f6d-ffc25b9169a1 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:06.360829 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 238/492] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:35:06.360829 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0025cec-ace0-405c-8529-c121dc9510a0 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:06.387796 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e6304-cb2f-4761-be87-2bfef118906e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:06.389027 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e6304-cb2f-4761-be87-2bfef118906e None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:06.389726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e6304-cb2f-4761-be87-2bfef118906e None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:06.390146 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e6304-cb2f-4761-be87-2bfef118906e None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:06.391464 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e6304-cb2f-4761-be87-2bfef118906e None admin] RBAC: Adding query filter params (name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:06.392231 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e6304-cb2f-4761-be87-2bfef118906e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:06.394475 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e6304-cb2f-4761-be87-2bfef118906e None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:06.410191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 255/493] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Aug 26 00:35:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:35:06.411341 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e6304-cb2f-4761-be87-2bfef118906e None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:06.461262 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:06.482217 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0025cec-ace0-405c-8529-c121dc9510a0 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:06.483876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:06.522165 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacfa649-4f9f-40fe-a256-39b44c24dba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:06.522800 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacfa649-4f9f-40fe-a256-39b44c24dba8 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:06.523082 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacfa649-4f9f-40fe-a256-39b44c24dba8 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:06.523353 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacfa649-4f9f-40fe-a256-39b44c24dba8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:06.536657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacfa649-4f9f-40fe-a256-39b44c24dba8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:06.541896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-bacfa649-4f9f-40fe-a256-39b44c24dba8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:06.541896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:06.541896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:06.541896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:06.541896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:06.541896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:06.541896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:06.541896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:06.541896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:06.541896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:06.541896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:06.541896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:06.541896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:06.541896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:06.541896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:06.541896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:06.543317 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:06.543317 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:06.543317 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:06.543317 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:06.543317 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:06.543317 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:06.543317 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:06.543317 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:06.543317 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:06.543317 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:06.543317 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:06.543317 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:06.543317 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:06.543317 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:06.543317 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:06.543317 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:06.544319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:06.544319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:06.544319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:06.544319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:06.544319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:06.544319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:06.544319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:06.544319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:06.544319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:06.544319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:06.544319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:06.544319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:06.544319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 239/494] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Aug 26 00:35:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:06.544319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacfa649-4f9f-40fe-a256-39b44c24dba8 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:06.554797 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e6304-cb2f-4761-be87-2bfef118906e None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:06.591879 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba7526-2e18-4f0d-b51e-74475db4b013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:06.592520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba7526-2e18-4f0d-b51e-74475db4b013 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:06.592806 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba7526-2e18-4f0d-b51e-74475db4b013 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:06.593080 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba7526-2e18-4f0d-b51e-74475db4b013 None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:06.593725 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba7526-2e18-4f0d-b51e-74475db4b013 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:06.594053 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba7526-2e18-4f0d-b51e-74475db4b013 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:06.595478 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba7526-2e18-4f0d-b51e-74475db4b013 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:06.606482 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 256/495] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:06.607664 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45e194-f46e-4591-871b-0289062b09ac None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:06.607942 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45e194-f46e-4591-871b-0289062b09ac None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:06.608213 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45e194-f46e-4591-871b-0289062b09ac None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:06.612522 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 257/496] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:06.612522 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45e194-f46e-4591-871b-0289062b09ac None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:06.660113 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacfa649-4f9f-40fe-a256-39b44c24dba8 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:06.692427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a8b1f-2565-47f6-804b-c6798411e864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:06.693075 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a8b1f-2565-47f6-804b-c6798411e864 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:06.693361 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a8b1f-2565-47f6-804b-c6798411e864 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:06.693621 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a8b1f-2565-47f6-804b-c6798411e864 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:06.701741 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:06.704008 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450a8b1f-2565-47f6-804b-c6798411e864 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:06.705490 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450a8b1f-2565-47f6-804b-c6798411e864 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:06.710917 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45e194-f46e-4591-871b-0289062b09ac None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:06.720323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-450a8b1f-2565-47f6-804b-c6798411e864 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:06.720323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:06.720323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:06.720323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:06.720323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:06.720323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:06.720323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:06.720323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:06.720323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:06.720323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:06.720323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:06.720323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:06.720323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:06.720323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:06.720323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:06.720323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:06.727957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:06.727957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:06.727957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:06.727957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:06.727957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:06.727957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:06.727957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:06.727957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:06.727957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:06.727957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:06.727957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:06.727957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:06.727957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:06.727957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:06.727957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:06.727957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:06.727957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:06.729044 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:06.729044 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:06.729044 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:06.729044 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 240/497] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:06.729044 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a8b1f-2565-47f6-804b-c6798411e864 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:06.737561 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:06.751867 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c3c1f-b26c-4c07-b9e6-8c33a86319a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:06.759040 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c3c1f-b26c-4c07-b9e6-8c33a86319a0 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:06.759040 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c3c1f-b26c-4c07-b9e6-8c33a86319a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:06.759040 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c3c1f-b26c-4c07-b9e6-8c33a86319a0 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:06.759040 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c3c1f-b26c-4c07-b9e6-8c33a86319a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:06.759040 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c3c1f-b26c-4c07-b9e6-8c33a86319a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:06.759040 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c3c1f-b26c-4c07-b9e6-8c33a86319a0 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:06.763310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 258/498] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Aug 26 00:35:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:06.764465 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc685a-81b6-429d-9cfc-2a662a53fa3f None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:06.764782 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc685a-81b6-429d-9cfc-2a662a53fa3f None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:06.765097 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc685a-81b6-429d-9cfc-2a662a53fa3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:06.783243 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-5cdc685a-81b6-429d-9cfc-2a662a53fa3f None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:06.784420 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-5cdc685a-81b6-429d-9cfc-2a662a53fa3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:06.819505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a8b1f-2565-47f6-804b-c6798411e864 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:06.849703 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 259/499] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:35:06.850319 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc685a-81b6-429d-9cfc-2a662a53fa3f None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:06.853796 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc823c3e-6612-480a-875d-172045a49a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:06.854427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc823c3e-6612-480a-875d-172045a49a10 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:06.854707 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc823c3e-6612-480a-875d-172045a49a10 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:06.854979 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc823c3e-6612-480a-875d-172045a49a10 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:06.862952 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc823c3e-6612-480a-875d-172045a49a10 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:06.864498 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc823c3e-6612-480a-875d-172045a49a10 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:06.878511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-dc823c3e-6612-480a-875d-172045a49a10 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 00:35:06.878511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:06.878511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:06.878511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:06.878511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:06.878511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:06.878511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:06.878511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:06.878511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:06.878511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:06.878511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:06.878511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:06.878511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:35:06.878511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:35:06.878511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:35:06.878511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:35:06.879432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:06.879432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:06.879432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:06.879432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:06.879432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:06.879432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:06.879432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:06.879432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:06.879432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:06.879432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:06.879432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:06.879432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:06.879432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:06.879432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:06.879432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:06.879432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:06.879432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:06.879432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:06.880370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:06.880370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:06.880370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:35:06.880370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:35:06.880370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:35:06.880370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:35:06.880370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:35:06.880370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:35:06.880370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 00:35:06.880370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:06.880370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 241/500] 10.222.0.59 () {56 vars in 1093 bytes} [Fri Aug 26 00:35:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:06.881000 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc823c3e-6612-480a-875d-172045a49a10 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:06.944689 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:06.966162 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc685a-81b6-429d-9cfc-2a662a53fa3f None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:06.970585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:06.996275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc823c3e-6612-480a-875d-172045a49a10 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:07.008642 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098426a-6022-40e1-8770-416f62cff5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:07.009442 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098426a-6022-40e1-8770-416f62cff5b2 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:07.009721 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098426a-6022-40e1-8770-416f62cff5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:07.010001 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098426a-6022-40e1-8770-416f62cff5b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:07.022976 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5098426a-6022-40e1-8770-416f62cff5b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:07.027935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-5098426a-6022-40e1-8770-416f62cff5b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:07.027935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:07.027935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:07.027935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:07.027935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:07.027935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:07.027935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:07.027935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:07.027935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:07.027935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:07.027935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:07.027935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:07.027935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:07.027935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:07.027935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:07.027935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:07.029293 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:07.029293 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:07.029293 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:07.029293 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:07.029293 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:07.029293 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:07.029293 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:07.029293 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:07.029293 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:07.029293 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:07.029293 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:07.029293 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:07.029293 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:07.029293 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:07.029293 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:07.029293 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:07.039972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:07.039972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:07.039972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:07.039972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:07.039972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:07.039972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:07.039972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:07.039972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:07.039972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:07.039972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:07.039972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:07.039972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:07.039972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 260/501] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Aug 26 00:35:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:07.039972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098426a-6022-40e1-8770-416f62cff5b2 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:07.040943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6bb03-fab0-4040-bf12-6a85eeac4f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:07.040943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6bb03-fab0-4040-bf12-6a85eeac4f62 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:07.040943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6bb03-fab0-4040-bf12-6a85eeac4f62 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:07.040943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6bb03-fab0-4040-bf12-6a85eeac4f62 None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:07.041217 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6bb03-fab0-4040-bf12-6a85eeac4f62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:07.041217 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6bb03-fab0-4040-bf12-6a85eeac4f62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:07.041217 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6bb03-fab0-4040-bf12-6a85eeac4f62 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:07.050649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 242/502] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:07.051395 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6bb03-fab0-4040-bf12-6a85eeac4f62 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:07.139542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098426a-6022-40e1-8770-416f62cff5b2 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:07.152643 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6bb03-fab0-4040-bf12-6a85eeac4f62 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:07.175027 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049760bd-cae9-4209-b036-28fa6a58edef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:07.175657 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049760bd-cae9-4209-b036-28fa6a58edef None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:07.175934 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049760bd-cae9-4209-b036-28fa6a58edef None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:07.176203 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049760bd-cae9-4209-b036-28fa6a58edef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:07.185980 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049760bd-cae9-4209-b036-28fa6a58edef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:07.187219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5acb9-1b34-49c3-9720-6b84e9c53877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:07.187698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049760bd-cae9-4209-b036-28fa6a58edef None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:07.187864 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5acb9-1b34-49c3-9720-6b84e9c53877 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:07.188139 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5acb9-1b34-49c3-9720-6b84e9c53877 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:07.188407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5acb9-1b34-49c3-9720-6b84e9c53877 None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:07.189166 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5acb9-1b34-49c3-9720-6b84e9c53877 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:07.189495 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5acb9-1b34-49c3-9720-6b84e9c53877 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:07.189995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:07.191017 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5acb9-1b34-49c3-9720-6b84e9c53877 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:07.192929 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:07.202400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-049760bd-cae9-4209-b036-28fa6a58edef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:07.202400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:07.202400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:07.202400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:07.202400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:07.202400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:07.202400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:07.202400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:07.202400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:07.202400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:07.202400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:07.202400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:07.202400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:07.202400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:07.202400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:07.202400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:07.207752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:07.207752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:07.207752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:07.207752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:07.207752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:07.207752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:07.207752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:07.207752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:07.207752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:07.207752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:07.207752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:07.207752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:07.207752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:07.207752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:07.207752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:07.207752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:07.207752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:07.208737 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:07.208737 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:07.208737 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:07.208737 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 261/503] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:07.208737 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049760bd-cae9-4209-b036-28fa6a58edef None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:07.228067 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 243/504] 10.222.0.59 () {56 vars in 1133 bytes} [Fri Aug 26 00:35:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:07.228945 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5acb9-1b34-49c3-9720-6b84e9c53877 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:07.309346 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049760bd-cae9-4209-b036-28fa6a58edef None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:07.328970 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5acb9-1b34-49c3-9720-6b84e9c53877 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:07.342769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed10b487-aec6-4138-ba25-8d2b7b6e78ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:07.343377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10b487-aec6-4138-ba25-8d2b7b6e78ea None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:07.343667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10b487-aec6-4138-ba25-8d2b7b6e78ea None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:07.343965 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10b487-aec6-4138-ba25-8d2b7b6e78ea None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:07.344662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed10b487-aec6-4138-ba25-8d2b7b6e78ea None admin] RBAC: Adding query filter params (name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:07.344969 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed10b487-aec6-4138-ba25-8d2b7b6e78ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:07.346353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed10b487-aec6-4138-ba25-8d2b7b6e78ea None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:07.354449 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 262/505] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Aug 26 00:35:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:07.355254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed10b487-aec6-4138-ba25-8d2b7b6e78ea None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:07.363448 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd6bbe-4992-4ab8-9143-bd8799ea6efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:07.364081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd6bbe-4992-4ab8-9143-bd8799ea6efc None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:07.364361 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd6bbe-4992-4ab8-9143-bd8799ea6efc None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:07.364664 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd6bbe-4992-4ab8-9143-bd8799ea6efc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:07.371514 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd6bbe-4992-4ab8-9143-bd8799ea6efc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:07.372990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd6bbe-4992-4ab8-9143-bd8799ea6efc None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:07.384685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-06cd6bbe-4992-4ab8-9143-bd8799ea6efc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 00:35:07.384685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:07.384685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:07.384685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:07.384685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:07.384685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:07.384685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:07.384685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:07.384685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:07.384685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:07.384685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:07.384685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:07.384685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:35:07.384685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:35:07.384685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:35:07.384685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:35:07.385767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:07.385767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:07.385767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:07.385767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:07.385767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:07.385767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:07.385767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:07.385767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:07.385767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:07.385767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:07.385767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:07.385767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:07.385767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:07.385767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:07.385767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:07.385767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:07.385767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:07.385767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:07.386686 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:07.386686 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:07.386686 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:35:07.386686 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:35:07.386686 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:35:07.386686 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:35:07.386686 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:35:07.386686 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:35:07.386686 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 00:35:07.386686 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:07.386686 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 244/506] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:35:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:07.386686 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd6bbe-4992-4ab8-9143-bd8799ea6efc None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:07.418145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:07.439341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:07.461730 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed10b487-aec6-4138-ba25-8d2b7b6e78ea None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:07.486166 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd6bbe-4992-4ab8-9143-bd8799ea6efc None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:07.495718 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addad790-261e-4903-a5c3-b116decc12b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:07.496351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addad790-261e-4903-a5c3-b116decc12b9 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:07.496659 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addad790-261e-4903-a5c3-b116decc12b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:07.496934 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addad790-261e-4903-a5c3-b116decc12b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:07.507963 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addad790-261e-4903-a5c3-b116decc12b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:07.509454 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addad790-261e-4903-a5c3-b116decc12b9 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:07.519812 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0f921-d4a0-46ae-84b2-3ccaa09ca511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:07.520408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0f921-d4a0-46ae-84b2-3ccaa09ca511 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:07.520734 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0f921-d4a0-46ae-84b2-3ccaa09ca511 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:07.521002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0f921-d4a0-46ae-84b2-3ccaa09ca511 None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:07.521629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf0f921-d4a0-46ae-84b2-3ccaa09ca511 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:07.521923 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf0f921-d4a0-46ae-84b2-3ccaa09ca511 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:07.523267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf0f921-d4a0-46ae-84b2-3ccaa09ca511 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:07.523410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-addad790-261e-4903-a5c3-b116decc12b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:07.523410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:07.523410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:07.523410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:07.523410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:07.523410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:07.523410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:07.523410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:07.523410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:07.523410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:07.523410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:07.523410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:07.523410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:07.523410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:07.523410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:07.523410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:07.524426 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:07.524426 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:07.524426 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:07.524426 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:07.524426 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:07.524426 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:07.524426 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:07.524426 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:07.524426 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:07.524426 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:07.524426 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:07.524426 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:07.524426 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:07.524426 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:07.524426 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:07.524426 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:07.524426 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:07.525310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:07.525310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:07.525310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:07.525310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 263/507] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:07.525534 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addad790-261e-4903-a5c3-b116decc12b9 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:07.536625 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 245/508] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:07.537492 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0f921-d4a0-46ae-84b2-3ccaa09ca511 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:07.629272 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addad790-261e-4903-a5c3-b116decc12b9 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:07.633528 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0f921-d4a0-46ae-84b2-3ccaa09ca511 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:07.639580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:07.646637 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:07.664153 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e580104-e5b0-485f-853c-03fe300e18a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:07.664789 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e580104-e5b0-485f-853c-03fe300e18a2 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:07.665075 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e580104-e5b0-485f-853c-03fe300e18a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:07.665352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e580104-e5b0-485f-853c-03fe300e18a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:07.667094 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71746473-000e-4418-aaa9-986137c38c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:07.667716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71746473-000e-4418-aaa9-986137c38c98 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:07.668162 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71746473-000e-4418-aaa9-986137c38c98 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:07.668441 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71746473-000e-4418-aaa9-986137c38c98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:07.679073 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71746473-000e-4418-aaa9-986137c38c98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:07.680559 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71746473-000e-4418-aaa9-986137c38c98 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:07.694582 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-71746473-000e-4418-aaa9-986137c38c98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:07.694582 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:07.694582 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:07.694582 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:07.694582 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:07.694582 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:07.694582 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:07.694582 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:07.694582 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:07.694582 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:07.694582 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:07.694582 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:07.694582 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:07.694582 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:07.694582 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:07.694582 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:07.695491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:07.695491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:07.695491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:07.695491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:07.695491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:07.695491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:07.695491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:07.695491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:07.695491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:07.695491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:07.695491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:07.695491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:07.695491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:07.695491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:07.695491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:07.695491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:07.695491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:07.696340 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:07.696340 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:07.696340 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:07.696340 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 246/509] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:07.696609 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71746473-000e-4418-aaa9-986137c38c98 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:07.735639 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e580104-e5b0-485f-853c-03fe300e18a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:07.737125 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e580104-e5b0-485f-853c-03fe300e18a2 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:07.794985 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71746473-000e-4418-aaa9-986137c38c98 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:07.828411 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2622405-c962-4a72-9ccf-12fb694fa1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:07.829045 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2622405-c962-4a72-9ccf-12fb694fa1e1 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:07.829329 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2622405-c962-4a72-9ccf-12fb694fa1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:07.829606 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2622405-c962-4a72-9ccf-12fb694fa1e1 None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:07.830337 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2622405-c962-4a72-9ccf-12fb694fa1e1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:07.830648 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2622405-c962-4a72-9ccf-12fb694fa1e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:07.832086 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2622405-c962-4a72-9ccf-12fb694fa1e1 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:07.834673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 264/510] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:35:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:35:07.835520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e580104-e5b0-485f-853c-03fe300e18a2 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:07.846705 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:07.850952 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 247/511] 10.222.0.59 () {56 vars in 1137 bytes} [Fri Aug 26 00:35:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:07.851786 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2622405-c962-4a72-9ccf-12fb694fa1e1 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:07.876849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:07.933805 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e580104-e5b0-485f-853c-03fe300e18a2 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:07.948940 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2622405-c962-4a72-9ccf-12fb694fa1e1 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:07.966941 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed28ba-bf9f-4b06-8941-a35ed3b077ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:07.967543 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed28ba-bf9f-4b06-8941-a35ed3b077ef None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:07.967821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed28ba-bf9f-4b06-8941-a35ed3b077ef None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:07.968092 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed28ba-bf9f-4b06-8941-a35ed3b077ef None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:07.968800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed28ba-bf9f-4b06-8941-a35ed3b077ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:07.969132 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed28ba-bf9f-4b06-8941-a35ed3b077ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:07.970492 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed28ba-bf9f-4b06-8941-a35ed3b077ef None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:07.981246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 265/512] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:07.981546 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a8ac07-f680-4329-b89c-9d560ea57411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:07.982032 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed28ba-bf9f-4b06-8941-a35ed3b077ef None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:07.982198 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a8ac07-f680-4329-b89c-9d560ea57411 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:07.982442 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a8ac07-f680-4329-b89c-9d560ea57411 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:07.982713 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a8ac07-f680-4329-b89c-9d560ea57411 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:07.989604 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a8ac07-f680-4329-b89c-9d560ea57411 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:07.991037 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a8ac07-f680-4329-b89c-9d560ea57411 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:08.002604 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-33a8ac07-f680-4329-b89c-9d560ea57411 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 26 00:35:08.002604 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:08.002604 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:08.002604 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:08.002604 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:08.002604 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:08.002604 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:08.002604 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:08.002604 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:08.002604 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:08.002604 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:08.002604 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:08.002604 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:35:08.002604 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:35:08.002604 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:35:08.002604 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:35:08.003670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:08.003670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:08.003670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:08.003670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:08.003670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:08.003670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:08.003670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:08.003670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:08.003670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:08.003670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:08.003670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:08.003670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:08.003670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:08.003670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:08.003670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:08.003670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:08.003670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:08.003670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:08.009303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:08.009303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:08.009303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:35:08.009303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:35:08.009303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:35:08.009303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:35:08.009303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:35:08.009303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:35:08.009303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 26 00:35:08.009303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:08.009303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 248/513] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:35:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:08.009303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a8ac07-f680-4329-b89c-9d560ea57411 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:08.053765 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:08.084491 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed28ba-bf9f-4b06-8941-a35ed3b077ef None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:08.088285 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:08.106870 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a8ac07-f680-4329-b89c-9d560ea57411 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:08.117732 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c2f61-0a43-451f-98f6-20e4c53f0bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:08.118342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c2f61-0a43-451f-98f6-20e4c53f0bd8 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:08.118627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c2f61-0a43-451f-98f6-20e4c53f0bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:08.118902 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c2f61-0a43-451f-98f6-20e4c53f0bd8 None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:08.119547 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c2f61-0a43-451f-98f6-20e4c53f0bd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:08.119854 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c2f61-0a43-451f-98f6-20e4c53f0bd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:08.121251 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c2f61-0a43-451f-98f6-20e4c53f0bd8 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:08.133237 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 266/514] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:08.134030 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c2f61-0a43-451f-98f6-20e4c53f0bd8 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:08.146756 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8d069-2913-4c18-bc35-19da0d113d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:08.147388 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8d069-2913-4c18-bc35-19da0d113d12 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:08.147670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8d069-2913-4c18-bc35-19da0d113d12 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:08.147945 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8d069-2913-4c18-bc35-19da0d113d12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:08.158545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8d069-2913-4c18-bc35-19da0d113d12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:08.159977 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8d069-2913-4c18-bc35-19da0d113d12 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:08.174468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-3bf8d069-2913-4c18-bc35-19da0d113d12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:08.174468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:08.174468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:08.174468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:08.174468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:08.174468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:08.174468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:08.174468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:08.174468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:08.174468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:08.174468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:08.174468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:08.174468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:08.174468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:08.174468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:08.174468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:08.175418 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:08.175418 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:08.175418 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:08.175418 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:08.175418 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:08.175418 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:08.175418 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:08.175418 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:08.175418 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:08.175418 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:08.175418 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:08.175418 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:08.175418 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:08.175418 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:08.175418 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:08.175418 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:08.175418 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:08.176283 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:08.176283 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:08.176283 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:08.176283 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 249/515] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:08.176283 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8d069-2913-4c18-bc35-19da0d113d12 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:08.236356 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c2f61-0a43-451f-98f6-20e4c53f0bd8 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:08.263618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:08.269642 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e87c44-4184-4b00-a3ee-0edd0e41e494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:08.270248 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e87c44-4184-4b00-a3ee-0edd0e41e494 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:08.270530 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e87c44-4184-4b00-a3ee-0edd0e41e494 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:08.270801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e87c44-4184-4b00-a3ee-0edd0e41e494 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:08.272691 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8d069-2913-4c18-bc35-19da0d113d12 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:08.305739 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6b5e6-7d14-412a-8167-db0d9be9f8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:08.306338 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6b5e6-7d14-412a-8167-db0d9be9f8f3 None admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:08.306617 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6b5e6-7d14-412a-8167-db0d9be9f8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:08.306891 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6b5e6-7d14-412a-8167-db0d9be9f8f3 None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:08.307526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6b5e6-7d14-412a-8167-db0d9be9f8f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:08.309010 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6b5e6-7d14-412a-8167-db0d9be9f8f3 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:08.316725 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:08.337832 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e87c44-4184-4b00-a3ee-0edd0e41e494 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:08.339268 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e87c44-4184-4b00-a3ee-0edd0e41e494 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:08.392147 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 250/516] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Aug 26 00:35:08 2022] POST /identity/v3/users => generated 247 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:35:08.393005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6b5e6-7d14-412a-8167-db0d9be9f8f3 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:08.459771 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 267/517] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:35:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:35:08.459771 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e87c44-4184-4b00-a3ee-0edd0e41e494 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:08.490793 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:08.524690 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6b5e6-7d14-412a-8167-db0d9be9f8f3 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:08.561829 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b63679-2bae-454d-bf56-4bd3472f5b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:08.562482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b63679-2bae-454d-bf56-4bd3472f5b54 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:08.563270 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b63679-2bae-454d-bf56-4bd3472f5b54 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:08.563545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b63679-2bae-454d-bf56-4bd3472f5b54 None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:08.564518 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b63679-2bae-454d-bf56-4bd3472f5b54 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:08.564625 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b63679-2bae-454d-bf56-4bd3472f5b54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:08.566028 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b63679-2bae-454d-bf56-4bd3472f5b54 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:08.578770 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e87c44-4184-4b00-a3ee-0edd0e41e494 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:08.587082 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 251/518] 10.222.0.59 () {56 vars in 1137 bytes} [Fri Aug 26 00:35:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:08.588139 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b63679-2bae-454d-bf56-4bd3472f5b54 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:08.619836 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a75c15-a081-4a87-a033-eb033a364ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:08.620616 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a75c15-a081-4a87-a033-eb033a364ec9 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:08.621164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a75c15-a081-4a87-a033-eb033a364ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:08.621273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a75c15-a081-4a87-a033-eb033a364ec9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:08.628611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a75c15-a081-4a87-a033-eb033a364ec9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:08.630078 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a75c15-a081-4a87-a033-eb033a364ec9 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:08.640511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:08.644141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-71a75c15-a081-4a87-a033-eb033a364ec9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 00:35:08.644141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:08.644141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:08.644141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:08.644141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:08.644141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:08.644141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:08.644141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:08.644141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:08.644141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:08.644141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:08.644141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:08.644141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:35:08.644141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:35:08.644141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:35:08.644141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:35:08.645085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:08.645085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:08.645085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:08.645085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:08.645085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:08.645085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:08.645085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:08.645085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:08.645085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:08.645085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:08.645085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:08.645085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:08.645085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:08.645085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:08.645085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:08.645085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:08.645085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:08.645085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:08.646005 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:08.646005 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:08.646005 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:35:08.646005 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:35:08.646005 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:35:08.646005 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:35:08.646005 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:35:08.646005 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:35:08.646005 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 00:35:08.646005 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:08.646005 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 268/519] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Aug 26 00:35:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:08.646576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a75c15-a081-4a87-a033-eb033a364ec9 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:08.697818 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b63679-2bae-454d-bf56-4bd3472f5b54 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:08.731490 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca64dc-87da-40f2-adb7-f094fede752e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:08.732102 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca64dc-87da-40f2-adb7-f094fede752e None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:08.732384 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca64dc-87da-40f2-adb7-f094fede752e None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:08.732686 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca64dc-87da-40f2-adb7-f094fede752e None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:08.733331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca64dc-87da-40f2-adb7-f094fede752e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:08.733637 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca64dc-87da-40f2-adb7-f094fede752e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:08.735037 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca64dc-87da-40f2-adb7-f094fede752e None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:08.746014 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:08.747079 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 252/520] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:08.748446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca64dc-87da-40f2-adb7-f094fede752e None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:08.761076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a75c15-a081-4a87-a033-eb033a364ec9 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:08.799837 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f5963-f42b-47ed-98d0-a785e04d2c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:08.800464 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f5963-f42b-47ed-98d0-a785e04d2c5f None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:08.800765 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f5963-f42b-47ed-98d0-a785e04d2c5f None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:08.801037 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f5963-f42b-47ed-98d0-a785e04d2c5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:08.811751 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f5963-f42b-47ed-98d0-a785e04d2c5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:08.814827 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f5963-f42b-47ed-98d0-a785e04d2c5f None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:08.837497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-b52f5963-f42b-47ed-98d0-a785e04d2c5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:08.837497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:08.837497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:08.837497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:08.837497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:08.837497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:08.837497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:08.837497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:08.837497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:08.837497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:08.837497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:08.837497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:08.837497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:08.837497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:08.837497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:08.837497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:08.838548 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:08.838548 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:08.838548 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:08.838548 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:08.838548 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:08.838548 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:08.838548 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:08.838548 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:08.838548 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:08.838548 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:08.838548 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:08.838548 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:08.838548 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:08.838548 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:08.838548 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:08.838548 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:08.838548 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:08.839542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:08.839542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:08.839542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:08.839542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 269/521] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:08.840345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f5963-f42b-47ed-98d0-a785e04d2c5f None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:08.860416 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca64dc-87da-40f2-adb7-f094fede752e None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:08.870905 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:08.903662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b521201a-3537-4ef3-b29b-bc4f6c53a3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:08.904651 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521201a-3537-4ef3-b29b-bc4f6c53a3d7 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:08.905138 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521201a-3537-4ef3-b29b-bc4f6c53a3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:08.905589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521201a-3537-4ef3-b29b-bc4f6c53a3d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:08.978916 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f5963-f42b-47ed-98d0-a785e04d2c5f None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:08.997617 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b521201a-3537-4ef3-b29b-bc4f6c53a3d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:08.999470 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b521201a-3537-4ef3-b29b-bc4f6c53a3d7 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:09.014385 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeefbd9-1e43-4400-be2a-1a3dd8d291e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:09.014999 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeefbd9-1e43-4400-be2a-1a3dd8d291e5 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:09.015284 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeefbd9-1e43-4400-be2a-1a3dd8d291e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:09.016101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeefbd9-1e43-4400-be2a-1a3dd8d291e5 None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:09.016944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeefbd9-1e43-4400-be2a-1a3dd8d291e5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:09.017252 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeefbd9-1e43-4400-be2a-1a3dd8d291e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:09.018673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeefbd9-1e43-4400-be2a-1a3dd8d291e5 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:09.053641 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:09.057284 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 270/522] 10.222.0.59 () {56 vars in 1139 bytes} [Fri Aug 26 00:35:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:09.058176 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeefbd9-1e43-4400-be2a-1a3dd8d291e5 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:09.113760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 253/523] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:35:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:35:09.115169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b521201a-3537-4ef3-b29b-bc4f6c53a3d7 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:09.125858 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:09.172496 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeefbd9-1e43-4400-be2a-1a3dd8d291e5 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:09.218366 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6128adfd-8407-4200-b2de-b98de6c7fde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:09.219527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128adfd-8407-4200-b2de-b98de6c7fde4 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:09.220138 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128adfd-8407-4200-b2de-b98de6c7fde4 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:09.220592 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128adfd-8407-4200-b2de-b98de6c7fde4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:09.226348 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b521201a-3537-4ef3-b29b-bc4f6c53a3d7 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:09.229824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6128adfd-8407-4200-b2de-b98de6c7fde4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:09.231483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6128adfd-8407-4200-b2de-b98de6c7fde4 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:09.246446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-6128adfd-8407-4200-b2de-b98de6c7fde4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:09.246446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:09.246446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:09.246446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:09.246446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:09.246446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:09.246446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:09.246446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:09.246446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:09.246446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:09.246446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:09.246446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:09.246446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:35:09.246446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:35:09.246446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:35:09.247536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:35:09.247536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:09.247536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:09.247536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:09.247536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:09.247536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:09.247536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:09.247536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:09.247536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:09.247536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:09.247536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:09.247536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:09.247536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:09.247536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:09.247536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:09.247536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:09.247536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:09.248668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:35:09.248668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:35:09.248668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:35:09.248668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:35:09.248668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:35:09.248668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:35:09.248668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:09.248668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:09.254832 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 271/524] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:09.254832 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6128adfd-8407-4200-b2de-b98de6c7fde4 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:09.269142 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d48261-4cc1-4549-a4e0-06e01e7de95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:09.269753 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d48261-4cc1-4549-a4e0-06e01e7de95c None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:09.270031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d48261-4cc1-4549-a4e0-06e01e7de95c None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:09.270302 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d48261-4cc1-4549-a4e0-06e01e7de95c None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:09.271058 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d48261-4cc1-4549-a4e0-06e01e7de95c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:09.271363 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d48261-4cc1-4549-a4e0-06e01e7de95c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:09.273640 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d48261-4cc1-4549-a4e0-06e01e7de95c None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:09.286375 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 254/525] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:09.287197 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d48261-4cc1-4549-a4e0-06e01e7de95c None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:09.320573 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:09.358352 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:09.362062 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6128adfd-8407-4200-b2de-b98de6c7fde4 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:09.393445 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d48261-4cc1-4549-a4e0-06e01e7de95c None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:09.397891 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2965681-90a6-4983-86d3-e3fe9cdd8691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:09.398516 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2965681-90a6-4983-86d3-e3fe9cdd8691 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:09.398828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2965681-90a6-4983-86d3-e3fe9cdd8691 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:09.399108 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2965681-90a6-4983-86d3-e3fe9cdd8691 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:09.433835 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690fab7-c03a-4306-9e93-b3e16e8f86a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:09.434477 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690fab7-c03a-4306-9e93-b3e16e8f86a0 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:09.434753 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690fab7-c03a-4306-9e93-b3e16e8f86a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:09.435021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690fab7-c03a-4306-9e93-b3e16e8f86a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:09.446764 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e690fab7-c03a-4306-9e93-b3e16e8f86a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:09.448199 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e690fab7-c03a-4306-9e93-b3e16e8f86a0 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:09.461968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-e690fab7-c03a-4306-9e93-b3e16e8f86a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:09.461968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:09.461968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:09.461968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:09.461968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:09.461968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:09.461968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:09.461968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:09.461968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:09.461968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:09.461968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:09.461968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:09.461968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:09.461968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:09.461968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:09.461968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:09.463002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:09.463002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:09.463002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:09.463002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:09.463002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:09.463002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:09.463002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:09.463002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:09.463002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:09.463002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:09.463002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:09.463002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:09.463002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:09.463002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:09.463002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:09.463002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:09.463002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:09.463887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:09.463887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:09.463887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:09.463887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 255/526] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:09.463887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690fab7-c03a-4306-9e93-b3e16e8f86a0 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:09.491487 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2965681-90a6-4983-86d3-e3fe9cdd8691 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:09.493459 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2965681-90a6-4983-86d3-e3fe9cdd8691 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:09.557128 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:09.574524 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690fab7-c03a-4306-9e93-b3e16e8f86a0 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:09.597696 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:09.605266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 272/527] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:35:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:35:09.605266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2965681-90a6-4983-86d3-e3fe9cdd8691 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:09.617094 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed670cf8-8c6b-42b4-9f5c-766b24e85ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:09.617704 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed670cf8-8c6b-42b4-9f5c-766b24e85ba1 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:09.618033 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed670cf8-8c6b-42b4-9f5c-766b24e85ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:09.618309 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed670cf8-8c6b-42b4-9f5c-766b24e85ba1 None admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:09.619016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed670cf8-8c6b-42b4-9f5c-766b24e85ba1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:09.619556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed670cf8-8c6b-42b4-9f5c-766b24e85ba1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:09.620741 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed670cf8-8c6b-42b4-9f5c-766b24e85ba1 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:09.639296 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 256/528] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:35:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:09.640112 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed670cf8-8c6b-42b4-9f5c-766b24e85ba1 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:09.712078 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2965681-90a6-4983-86d3-e3fe9cdd8691 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:09.746574 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed670cf8-8c6b-42b4-9f5c-766b24e85ba1 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:09.747972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8295d462-2893-4766-877d-56ea104c01c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:09.748641 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8295d462-2893-4766-877d-56ea104c01c9 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:09.748926 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8295d462-2893-4766-877d-56ea104c01c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:09.749206 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8295d462-2893-4766-877d-56ea104c01c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:09.756433 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8295d462-2893-4766-877d-56ea104c01c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:09.757919 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8295d462-2893-4766-877d-56ea104c01c9 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:09.769628 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-8295d462-2893-4766-877d-56ea104c01c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:09.769628 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:09.769628 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:09.769628 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:09.769628 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:09.769628 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:09.769628 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:09.769628 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:09.769628 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:09.769628 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:09.769628 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:09.769628 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:09.769628 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:35:09.769628 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:35:09.769628 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:35:09.770520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:35:09.770520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:09.770520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:09.770520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:09.770520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:09.770520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:09.770520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:09.770520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:09.770520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:09.770520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:09.770520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:09.770520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:09.770520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:09.770520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:09.770520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:09.770520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:09.770520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:09.771358 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:35:09.771358 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:35:09.771358 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:35:09.771358 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:35:09.771358 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:35:09.771358 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:35:09.771358 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:09.771358 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:09.771358 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 273/529] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:09.772455 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8295d462-2893-4766-877d-56ea104c01c9 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:09.791626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675fd313-7140-41c8-85ea-66123651e488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:09.792250 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675fd313-7140-41c8-85ea-66123651e488 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:09.792568 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675fd313-7140-41c8-85ea-66123651e488 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:09.792847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675fd313-7140-41c8-85ea-66123651e488 None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:09.793489 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675fd313-7140-41c8-85ea-66123651e488 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:09.793796 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675fd313-7140-41c8-85ea-66123651e488 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:09.795195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675fd313-7140-41c8-85ea-66123651e488 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:09.797310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:09.813110 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 257/530] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:09.813720 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675fd313-7140-41c8-85ea-66123651e488 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:09.858317 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:09.882370 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8295d462-2893-4766-877d-56ea104c01c9 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:09.918741 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675fd313-7140-41c8-85ea-66123651e488 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:09.921014 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d767c6-52dd-41e2-9e2f-f93910b0ded2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:09.921642 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d767c6-52dd-41e2-9e2f-f93910b0ded2 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:09.922428 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d767c6-52dd-41e2-9e2f-f93910b0ded2 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:09.922695 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d767c6-52dd-41e2-9e2f-f93910b0ded2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:09.933087 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d767c6-52dd-41e2-9e2f-f93910b0ded2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:09.934529 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d767c6-52dd-41e2-9e2f-f93910b0ded2 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:09.949603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-51d767c6-52dd-41e2-9e2f-f93910b0ded2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:09.949603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:09.949603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:09.949603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:09.949603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:09.949603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:09.949603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:09.949603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:09.949603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:09.949603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:09.949603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:09.949603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:09.949603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:09.949603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:09.949603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:09.949603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:09.950590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:09.950590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:09.950590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:09.950590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:09.950590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:09.950590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:09.950590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:09.950590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:09.950590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:09.950590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:09.950590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:09.950590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:09.950590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:09.950590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:09.950590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:09.950590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:09.950590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:09.951495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:09.951495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:09.951495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:09.951495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 274/531] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:09.952543 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d767c6-52dd-41e2-9e2f-f93910b0ded2 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:09.957348 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a3039-ff4d-41be-b890-0611467f444c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:09.957951 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a3039-ff4d-41be-b890-0611467f444c None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:09.958227 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a3039-ff4d-41be-b890-0611467f444c None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:09.958497 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a3039-ff4d-41be-b890-0611467f444c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:09.959218 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387a3039-ff4d-41be-b890-0611467f444c None admin] RBAC: Adding query filter params (role.id=bce78c7e0a534135a820de7f3588aa4d, scope.project.id=fde7ac3ed472433ab3bba56e5e091510, user.id=0602593263ca4b05a657d5b7697f932f) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:09.959522 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387a3039-ff4d-41be-b890-0611467f444c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:09.961000 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387a3039-ff4d-41be-b890-0611467f444c None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:09.976625 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 258/532] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:35:09 2022] GET /identity/v3/role_assignments?user.id=0602593263ca4b05a657d5b7697f932f&scope.project.id=fde7ac3ed472433ab3bba56e5e091510&role.id=bce78c7e0a534135a820de7f3588aa4d => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:09.976625 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a3039-ff4d-41be-b890-0611467f444c None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:10.035417 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:10.070497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d767c6-52dd-41e2-9e2f-f93910b0ded2 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:10.113502 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a3039-ff4d-41be-b890-0611467f444c None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:10.116333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:10.120555 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30643ea-f4a0-48f2-b214-63adbdea2dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:10.121197 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30643ea-f4a0-48f2-b214-63adbdea2dcc None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:10.121482 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30643ea-f4a0-48f2-b214-63adbdea2dcc None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:10.121838 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30643ea-f4a0-48f2-b214-63adbdea2dcc None admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:10.123252 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30643ea-f4a0-48f2-b214-63adbdea2dcc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:10.123582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30643ea-f4a0-48f2-b214-63adbdea2dcc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:10.125708 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30643ea-f4a0-48f2-b214-63adbdea2dcc None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:10.151645 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 275/533] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:35:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:10.153187 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30643ea-f4a0-48f2-b214-63adbdea2dcc None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:10.159567 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bfc1d8-a4d0-4002-8cd4-056014b8d9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:10.160232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfc1d8-a4d0-4002-8cd4-056014b8d9ad None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:10.160541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfc1d8-a4d0-4002-8cd4-056014b8d9ad None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:10.160820 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfc1d8-a4d0-4002-8cd4-056014b8d9ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:10.169979 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bfc1d8-a4d0-4002-8cd4-056014b8d9ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:10.172362 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bfc1d8-a4d0-4002-8cd4-056014b8d9ad None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:10.192796 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-06bfc1d8-a4d0-4002-8cd4-056014b8d9ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:10.192796 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:10.192796 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:10.192796 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:10.192796 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:10.192796 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:10.192796 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:10.192796 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:10.192796 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:10.192796 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:10.192796 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:10.192796 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:10.192796 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:35:10.192796 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:35:10.192796 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:35:10.193699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:35:10.193699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:10.193699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:10.193699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:10.193699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:10.193699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:10.193699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:10.193699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:10.193699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:10.193699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:10.193699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:10.193699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:10.193699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:10.193699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:10.193699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:10.193699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:10.193699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:10.194978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:35:10.194978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:35:10.194978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:35:10.194978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:35:10.194978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:35:10.194978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:35:10.194978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:10.194978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:10.196555 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 259/534] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:10.197566 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bfc1d8-a4d0-4002-8cd4-056014b8d9ad None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:10.295017 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30643ea-f4a0-48f2-b214-63adbdea2dcc None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:10.313669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bfc1d8-a4d0-4002-8cd4-056014b8d9ad None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:10.328602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:10.334182 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4977f3b-6e0d-4f7f-acea-21c03fdd8dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:10.334809 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4977f3b-6e0d-4f7f-acea-21c03fdd8dc4 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:10.335085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4977f3b-6e0d-4f7f-acea-21c03fdd8dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:10.335358 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4977f3b-6e0d-4f7f-acea-21c03fdd8dc4 None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:10.338166 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4977f3b-6e0d-4f7f-acea-21c03fdd8dc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:10.338166 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4977f3b-6e0d-4f7f-acea-21c03fdd8dc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:10.338166 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4977f3b-6e0d-4f7f-acea-21c03fdd8dc4 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:10.350658 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 276/535] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:10.351867 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4977f3b-6e0d-4f7f-acea-21c03fdd8dc4 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:10.359805 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833255b8-60de-4f43-af1e-18ff8b84dbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:10.360804 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833255b8-60de-4f43-af1e-18ff8b84dbbf None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:10.361293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833255b8-60de-4f43-af1e-18ff8b84dbbf None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:10.361765 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833255b8-60de-4f43-af1e-18ff8b84dbbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:10.363091 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833255b8-60de-4f43-af1e-18ff8b84dbbf None admin] RBAC: Adding query filter params (role.id=bce78c7e0a534135a820de7f3588aa4d, scope.project.id=fde7ac3ed472433ab3bba56e5e091510, user.id=c88202614775418ba94f478f03431a9b) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:10.363630 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833255b8-60de-4f43-af1e-18ff8b84dbbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:10.365975 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833255b8-60de-4f43-af1e-18ff8b84dbbf None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:10.373828 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 260/536] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:35:10 2022] GET /identity/v3/role_assignments?user.id=c88202614775418ba94f478f03431a9b&scope.project.id=fde7ac3ed472433ab3bba56e5e091510&role.id=bce78c7e0a534135a820de7f3588aa4d => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:10.375414 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833255b8-60de-4f43-af1e-18ff8b84dbbf None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:10.409385 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:10.477178 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4977f3b-6e0d-4f7f-acea-21c03fdd8dc4 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:10.504579 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833255b8-60de-4f43-af1e-18ff8b84dbbf None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:10.517364 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7eda9-fe14-4526-9dbc-bf3f6a1cd85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:10.518012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7eda9-fe14-4526-9dbc-bf3f6a1cd85f None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:10.518295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7eda9-fe14-4526-9dbc-bf3f6a1cd85f None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:10.518574 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7eda9-fe14-4526-9dbc-bf3f6a1cd85f None admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:10.519298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d7eda9-fe14-4526-9dbc-bf3f6a1cd85f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:10.519607 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d7eda9-fe14-4526-9dbc-bf3f6a1cd85f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:10.521194 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d7eda9-fe14-4526-9dbc-bf3f6a1cd85f None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:10.542268 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 277/537] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:35:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:10.543692 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8febdc6-1161-4d4e-82df-6d3389c1cd41 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:10.548844 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8febdc6-1161-4d4e-82df-6d3389c1cd41 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:10.549204 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8febdc6-1161-4d4e-82df-6d3389c1cd41 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:10.555236 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 278/538] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:10.555236 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8febdc6-1161-4d4e-82df-6d3389c1cd41 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:10.555460 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd2de5-b663-48a6-b4ae-5a7da705872d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:10.555460 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd2de5-b663-48a6-b4ae-5a7da705872d None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:10.555628 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd2de5-b663-48a6-b4ae-5a7da705872d None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:10.555987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd2de5-b663-48a6-b4ae-5a7da705872d None admin] PATH_INFO: `/v3/projects/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:10.564348 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd2de5-b663-48a6-b4ae-5a7da705872d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:10.565907 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd2de5-b663-48a6-b4ae-5a7da705872d None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:10.577427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-3fcd2de5-b663-48a6-b4ae-5a7da705872d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:10.577427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:10.577427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:10.577427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:10.577427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:10.577427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:10.577427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:10.577427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:10.577427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:10.577427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:10.577427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:10.577427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:10.577427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:35:10.577427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:35:10.577427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:35:10.578887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:35:10.578887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:10.578887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:10.578887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:10.578887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:10.578887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:10.578887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:10.578887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:10.578887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:10.578887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:10.578887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:10.578887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:10.578887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:10.578887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:10.578887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:10.578887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:10.578887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:10.579767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:35:10.579767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:35:10.579767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:35:10.579767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:35:10.579767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:35:10.579767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:35:10.579767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:10.579767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:10.579767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 261/539] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:10.580249 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a7ca5-760d-4cea-bfa4-0aa5045c8721 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:10.580790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a7ca5-760d-4cea-bfa4-0aa5045c8721 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:10.580790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a7ca5-760d-4cea-bfa4-0aa5045c8721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:10.600435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-473a7ca5-760d-4cea-bfa4-0aa5045c8721 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:10.601764 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-473a7ca5-760d-4cea-bfa4-0aa5045c8721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:10.629279 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:10.686257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 262/540] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:35:10.687551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a7ca5-760d-4cea-bfa4-0aa5045c8721 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:10.695399 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8febdc6-1161-4d4e-82df-6d3389c1cd41 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:10.725799 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:10.771519 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852b52f-d96b-4bde-a781-ecf0c6776939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:10.772140 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852b52f-d96b-4bde-a781-ecf0c6776939 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:10.772419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852b52f-d96b-4bde-a781-ecf0c6776939 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:10.772714 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852b52f-d96b-4bde-a781-ecf0c6776939 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:10.773451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852b52f-d96b-4bde-a781-ecf0c6776939 None admin] RBAC: Adding query filter params (role.id=bce78c7e0a534135a820de7f3588aa4d, scope.project.id=fde7ac3ed472433ab3bba56e5e091510, user.id=abed7ebb0aae4e4693ce08e51a149312) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:10.774856 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852b52f-d96b-4bde-a781-ecf0c6776939 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:10.776311 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852b52f-d96b-4bde-a781-ecf0c6776939 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:10.788980 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 279/541] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:35:10 2022] GET /identity/v3/role_assignments?user.id=abed7ebb0aae4e4693ce08e51a149312&scope.project.id=fde7ac3ed472433ab3bba56e5e091510&role.id=bce78c7e0a534135a820de7f3588aa4d => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:35:10.832075 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a7ca5-760d-4cea-bfa4-0aa5045c8721 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:10.868955 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df7baa-fbbd-43cd-b24b-774c455eb0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:10.870078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df7baa-fbbd-43cd-b24b-774c455eb0ec None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:10.870356 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df7baa-fbbd-43cd-b24b-774c455eb0ec None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:10.870669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df7baa-fbbd-43cd-b24b-774c455eb0ec None admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:10.871393 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df7baa-fbbd-43cd-b24b-774c455eb0ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:10.871696 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df7baa-fbbd-43cd-b24b-774c455eb0ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:10.873385 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df7baa-fbbd-43cd-b24b-774c455eb0ec None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:10.886908 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 263/542] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:35:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:10.892852 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852b52f-d96b-4bde-a781-ecf0c6776939 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:11.006232 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852b52f-d96b-4bde-a781-ecf0c6776939 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:11.022509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df7baa-fbbd-43cd-b24b-774c455eb0ec None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:11.027397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:11.049868 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff11c1-82ce-40df-b4b1-6d2e349addd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:11.050491 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff11c1-82ce-40df-b4b1-6d2e349addd9 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:11.050769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff11c1-82ce-40df-b4b1-6d2e349addd9 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:11.051040 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff11c1-82ce-40df-b4b1-6d2e349addd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:11.051777 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff11c1-82ce-40df-b4b1-6d2e349addd9 None admin] RBAC: Adding query filter params (role.id=bce78c7e0a534135a820de7f3588aa4d, scope.project.id=fde7ac3ed472433ab3bba56e5e091510, user.id=4698a49ed4a04f58a488359a2742e5da) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:11.052114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff11c1-82ce-40df-b4b1-6d2e349addd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:11.053632 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff11c1-82ce-40df-b4b1-6d2e349addd9 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:11.059811 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 280/543] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:35:10 2022] GET /identity/v3/role_assignments?user.id=4698a49ed4a04f58a488359a2742e5da&scope.project.id=fde7ac3ed472433ab3bba56e5e091510&role.id=bce78c7e0a534135a820de7f3588aa4d => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:11.118192 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:11.143817 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df7baa-fbbd-43cd-b24b-774c455eb0ec None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:11.188567 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cacc84-b8d6-4185-885f-dfa378037d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:11.189219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cacc84-b8d6-4185-885f-dfa378037d2a None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:11.189496 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cacc84-b8d6-4185-885f-dfa378037d2a None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:11.189765 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cacc84-b8d6-4185-885f-dfa378037d2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:11.206479 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cacc84-b8d6-4185-885f-dfa378037d2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:11.211750 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-c7cacc84-b8d6-4185-885f-dfa378037d2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:11.211750 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:11.211750 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:11.211750 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:11.211750 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:11.211750 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:11.211750 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:11.211750 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:11.211750 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:11.211750 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:11.211750 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:11.211750 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:11.211750 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:11.211750 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:11.211750 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:11.211750 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:11.212749 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:11.212749 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:11.212749 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:11.212749 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:11.212749 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:11.212749 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:11.212749 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:11.212749 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:11.212749 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:11.212749 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:11.212749 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:11.212749 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:11.212749 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:11.212749 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:11.212749 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:11.212749 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:11.213558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:11.213558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:11.213558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:11.213558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:11.213558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:11.213558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:11.213558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:11.213558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:11.213558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:11.213558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:11.213558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:11.213558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:11.213558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 264/544] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Aug 26 00:35:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:11.221797 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff11c1-82ce-40df-b4b1-6d2e349addd9 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:11.364599 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff11c1-82ce-40df-b4b1-6d2e349addd9 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:11.399262 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0172b7-7864-41bc-8402-e758dc440b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:11.399801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0172b7-7864-41bc-8402-e758dc440b05 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:11.400083 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0172b7-7864-41bc-8402-e758dc440b05 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:11.400358 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0172b7-7864-41bc-8402-e758dc440b05 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:11.407263 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0172b7-7864-41bc-8402-e758dc440b05 None admin] RBAC: Adding query filter params (name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:11.407581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0172b7-7864-41bc-8402-e758dc440b05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:11.409091 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0172b7-7864-41bc-8402-e758dc440b05 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:11.418179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 281/545] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Aug 26 00:35:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:11.423307 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cacc84-b8d6-4185-885f-dfa378037d2a None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:11.586281 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cacc84-b8d6-4185-885f-dfa378037d2a None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:11.625927 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5876e3-103d-4d99-b955-4087f83bba7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:11.626588 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5876e3-103d-4d99-b955-4087f83bba7b None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:11.626869 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5876e3-103d-4d99-b955-4087f83bba7b None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:11.627144 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5876e3-103d-4d99-b955-4087f83bba7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:11.635831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:11.644800 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5876e3-103d-4d99-b955-4087f83bba7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:11.646322 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5876e3-103d-4d99-b955-4087f83bba7b None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:11.661141 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-ed5876e3-103d-4d99-b955-4087f83bba7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:11.661141 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:11.661141 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:11.661141 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:11.661141 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:11.661141 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:11.661141 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:11.661141 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:11.661141 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:11.661141 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:11.661141 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:11.661141 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:11.661141 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:11.661141 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:11.661141 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:11.661141 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:11.662802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:11.662802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:11.662802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:11.662802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:11.662802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:11.662802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:11.662802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:11.662802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:11.662802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:11.662802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:11.662802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:11.662802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:11.662802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:11.662802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:11.662802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:11.662802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:11.662802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:11.663957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:11.663957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:11.663957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:11.663957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 265/546] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:11.669619 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0172b7-7864-41bc-8402-e758dc440b05 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:11.732248 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:11.810352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0172b7-7864-41bc-8402-e758dc440b05 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:11.856513 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8c436-b1e6-4222-a2ac-fa154dd40b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:11.857134 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8c436-b1e6-4222-a2ac-fa154dd40b12 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:11.857417 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8c436-b1e6-4222-a2ac-fa154dd40b12 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:11.857694 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8c436-b1e6-4222-a2ac-fa154dd40b12 None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:11.858342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf8c436-b1e6-4222-a2ac-fa154dd40b12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:11.858648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf8c436-b1e6-4222-a2ac-fa154dd40b12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:11.860068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf8c436-b1e6-4222-a2ac-fa154dd40b12 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:11.875732 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 282/547] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:11.881396 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5876e3-103d-4d99-b955-4087f83bba7b None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:12.016062 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5876e3-103d-4d99-b955-4087f83bba7b None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:12.043965 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef3f853-cc0b-4362-a79c-526db7d676a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:12.044633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3f853-cc0b-4362-a79c-526db7d676a8 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:12.053165 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3f853-cc0b-4362-a79c-526db7d676a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:12.053165 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3f853-cc0b-4362-a79c-526db7d676a8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:12.053462 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef3f853-cc0b-4362-a79c-526db7d676a8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:12.054962 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef3f853-cc0b-4362-a79c-526db7d676a8 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:12.066999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-0ef3f853-cc0b-4362-a79c-526db7d676a8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 00:35:12.066999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:12.066999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:12.066999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:12.066999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:12.066999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:12.066999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:12.066999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:12.066999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:12.066999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:12.066999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:12.066999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:12.066999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:35:12.066999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:35:12.066999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:35:12.066999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:35:12.068876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:12.068876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:12.068876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:12.068876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:12.068876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:12.068876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:12.068876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:12.068876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:12.068876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:12.068876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:12.068876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:12.068876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:12.068876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:12.068876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:12.068876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:12.068876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:12.068876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:12.068876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:12.070143 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:12.070143 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:12.070143 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:35:12.070143 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:35:12.070143 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:35:12.070143 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:35:12.070143 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:35:12.070143 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:35:12.070143 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 00:35:12.070143 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:12.070143 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 266/548] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:35:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:12.073748 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8c436-b1e6-4222-a2ac-fa154dd40b12 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:12.215116 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8c436-b1e6-4222-a2ac-fa154dd40b12 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:12.217007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:12.251410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9c246-a26e-4562-b46a-a34597a26e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:12.252870 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9c246-a26e-4562-b46a-a34597a26e17 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:12.253335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9c246-a26e-4562-b46a-a34597a26e17 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:12.253810 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9c246-a26e-4562-b46a-a34597a26e17 None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:12.254755 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f9c246-a26e-4562-b46a-a34597a26e17 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:12.255251 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f9c246-a26e-4562-b46a-a34597a26e17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:12.256990 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f9c246-a26e-4562-b46a-a34597a26e17 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:12.287762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 283/549] 10.222.0.59 () {56 vars in 1135 bytes} [Fri Aug 26 00:35:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:35:12.293624 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef3f853-cc0b-4362-a79c-526db7d676a8 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:12.376896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:12.438175 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef3f853-cc0b-4362-a79c-526db7d676a8 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:12.473748 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105dcb08-902f-463b-bf4c-c09e13d7caf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:12.474693 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105dcb08-902f-463b-bf4c-c09e13d7caf8 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:12.475200 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105dcb08-902f-463b-bf4c-c09e13d7caf8 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:12.475782 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105dcb08-902f-463b-bf4c-c09e13d7caf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:12.517254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147e38d-f3b6-4b53-add0-6ff303d39b41 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:12.517567 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147e38d-f3b6-4b53-add0-6ff303d39b41 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:12.517812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147e38d-f3b6-4b53-add0-6ff303d39b41 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:12.519400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 284/550] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:12.524414 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0a579-3f2c-4bf4-b6d1-bd3765c78fcc None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:12.524584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0a579-3f2c-4bf4-b6d1-bd3765c78fcc None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:12.525018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0a579-3f2c-4bf4-b6d1-bd3765c78fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:12.547623 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105dcb08-902f-463b-bf4c-c09e13d7caf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:12.549562 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105dcb08-902f-463b-bf4c-c09e13d7caf8 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:12.559592 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-a8c0a579-3f2c-4bf4-b6d1-bd3765c78fcc None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:12.560801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a8c0a579-3f2c-4bf4-b6d1-bd3765c78fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:12.603477 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:12.653578 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 267/551] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:35:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:35:12.661624 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105dcb08-902f-463b-bf4c-c09e13d7caf8 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:12.744933 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 285/552] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:35:12.788207 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105dcb08-902f-463b-bf4c-c09e13d7caf8 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:12.838349 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47a786-c611-4cee-8927-b6370166d9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:12.839004 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47a786-c611-4cee-8927-b6370166d9d3 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:12.839286 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47a786-c611-4cee-8927-b6370166d9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:12.839564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47a786-c611-4cee-8927-b6370166d9d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:12.853853 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47a786-c611-4cee-8927-b6370166d9d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:12.855301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47a786-c611-4cee-8927-b6370166d9d3 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:12.871786 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa8e02-60b6-4430-82a0-6a2b5d61d9a7 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:12.872392 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa8e02-60b6-4430-82a0-6a2b5d61d9a7 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:12.872903 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa8e02-60b6-4430-82a0-6a2b5d61d9a7 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:12.873920 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-fb47a786-c611-4cee-8927-b6370166d9d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:12.873920 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:12.873920 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:12.873920 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:12.873920 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:12.873920 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:12.873920 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:12.873920 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:12.873920 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:12.873920 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:12.873920 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:12.873920 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:12.873920 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:12.873920 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:12.873920 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:12.873920 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:12.874882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:12.874882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:12.874882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:12.874882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:12.874882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:12.874882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:12.874882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:12.874882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:12.874882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:12.874882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:12.874882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:12.874882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:12.874882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:12.874882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:12.874882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:12.874882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:12.874882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:12.875871 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:12.875871 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:12.875871 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:12.875871 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 268/553] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:12.877324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 286/554] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:35:12.880669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47a786-c611-4cee-8927-b6370166d9d3 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:12.883118 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39f838-5315-434d-bdfe-a70e1b806b6f None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:12.883831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39f838-5315-434d-bdfe-a70e1b806b6f None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:12.884312 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39f838-5315-434d-bdfe-a70e1b806b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:12.884786 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:12.914642 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-bd39f838-5315-434d-bdfe-a70e1b806b6f None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:12.917806 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-bd39f838-5315-434d-bdfe-a70e1b806b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:13.042718 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47a786-c611-4cee-8927-b6370166d9d3 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:13.060703 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 287/555] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:35:13.073676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd39f838-5315-434d-bdfe-a70e1b806b6f None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:13.087535 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d62cc6-cc0c-43dc-ae10-845f9ce84e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:13.088695 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d62cc6-cc0c-43dc-ae10-845f9ce84e08 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:13.089184 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d62cc6-cc0c-43dc-ae10-845f9ce84e08 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:13.089690 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d62cc6-cc0c-43dc-ae10-845f9ce84e08 None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:13.090768 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d62cc6-cc0c-43dc-ae10-845f9ce84e08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:13.091319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d62cc6-cc0c-43dc-ae10-845f9ce84e08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:13.092986 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d62cc6-cc0c-43dc-ae10-845f9ce84e08 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:13.100712 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:13.118817 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 269/556] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:13.125667 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d62cc6-cc0c-43dc-ae10-845f9ce84e08 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:13.214110 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd39f838-5315-434d-bdfe-a70e1b806b6f None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:13.247807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:13.268375 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daf78e4-302d-479f-8591-e9738fccb466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:13.269375 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf78e4-302d-479f-8591-e9738fccb466 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:13.269664 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf78e4-302d-479f-8591-e9738fccb466 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:13.270222 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf78e4-302d-479f-8591-e9738fccb466 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:13.284446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daf78e4-302d-479f-8591-e9738fccb466 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:13.291101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-9daf78e4-302d-479f-8591-e9738fccb466 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:13.291101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:13.291101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:13.291101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:13.291101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:13.291101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:13.291101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:13.291101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:13.291101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:13.291101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:13.291101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:13.291101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:13.291101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:13.291101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:13.291101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:13.291101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:13.292089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:13.292089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:13.292089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:13.292089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:13.292089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:13.292089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:13.292089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:13.292089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:13.292089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:13.292089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:13.292089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:13.292089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:13.292089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:13.292089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:13.292089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:13.292089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:13.292971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:13.292971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:13.292971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:13.292971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:13.292971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:13.292971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:13.292971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:13.292971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:13.292971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:13.292971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:13.292971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:13.292971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:13.292971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 288/557] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Aug 26 00:35:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:13.293681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d62cc6-cc0c-43dc-ae10-845f9ce84e08 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:13.294230 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed55b8-1efd-425a-bbb0-7d3c0502890e None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:13.294391 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed55b8-1efd-425a-bbb0-7d3c0502890e None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:13.300512 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed55b8-1efd-425a-bbb0-7d3c0502890e None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:13.301786 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 289/558] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:13.302624 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed55b8-1efd-425a-bbb0-7d3c0502890e None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:13.346170 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a0c07-f261-40f5-9ef1-a520a20fe1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:13.346731 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a0c07-f261-40f5-9ef1-a520a20fe1cc None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:13.347020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a0c07-f261-40f5-9ef1-a520a20fe1cc None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:13.347293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a0c07-f261-40f5-9ef1-a520a20fe1cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:13.357007 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a0c07-f261-40f5-9ef1-a520a20fe1cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:13.358459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a0c07-f261-40f5-9ef1-a520a20fe1cc None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:13.374259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-209a0c07-f261-40f5-9ef1-a520a20fe1cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:13.374259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:13.374259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:13.374259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:13.374259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:13.374259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:13.374259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:13.374259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:13.374259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:13.374259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:13.374259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:13.374259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:13.374259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:35:13.374259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:35:13.374259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:35:13.376224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:35:13.376224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:13.376224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:13.376224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:13.376224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:13.376224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:13.376224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:13.376224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:13.376224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:13.376224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:13.376224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:13.376224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:13.376224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:13.376224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:13.376224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:13.376224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:13.376224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:13.377162 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:35:13.377162 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:35:13.377162 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:35:13.377162 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:35:13.377162 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:35:13.377162 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:35:13.377162 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:13.377162 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:13.384425 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 270/559] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:13.384425 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b821fe0-de62-41ff-bcc7-e4051f8dcd99 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:13.384425 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b821fe0-de62-41ff-bcc7-e4051f8dcd99 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:13.384425 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b821fe0-de62-41ff-bcc7-e4051f8dcd99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:13.402386 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-5b821fe0-de62-41ff-bcc7-e4051f8dcd99 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:13.403890 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-5b821fe0-de62-41ff-bcc7-e4051f8dcd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:13.407866 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:13.428793 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed55b8-1efd-425a-bbb0-7d3c0502890e None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:13.463824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143677fe-edf0-4f6b-9c6e-43558febcdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:13.464436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143677fe-edf0-4f6b-9c6e-43558febcdb3 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:13.464879 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143677fe-edf0-4f6b-9c6e-43558febcdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:13.465283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143677fe-edf0-4f6b-9c6e-43558febcdb3 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:13.466174 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143677fe-edf0-4f6b-9c6e-43558febcdb3 None admin] RBAC: Adding query filter params (name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:13.466550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143677fe-edf0-4f6b-9c6e-43558febcdb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:13.468714 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143677fe-edf0-4f6b-9c6e-43558febcdb3 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:13.480175 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 271/560] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:35:13.480175 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137c028-25de-4b0b-801c-e1523634cd69 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:13.480175 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137c028-25de-4b0b-801c-e1523634cd69 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:13.480175 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137c028-25de-4b0b-801c-e1523634cd69 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:13.480175 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 272/561] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:13.480175 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137c028-25de-4b0b-801c-e1523634cd69 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:13.481036 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 290/562] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Aug 26 00:35:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:13.481439 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143677fe-edf0-4f6b-9c6e-43558febcdb3 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:13.549743 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:13.585844 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143677fe-edf0-4f6b-9c6e-43558febcdb3 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:13.603378 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137c028-25de-4b0b-801c-e1523634cd69 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:13.626277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12689705-e5d5-4273-a678-ed401776e0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:13.627202 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12689705-e5d5-4273-a678-ed401776e0bc None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:13.627622 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12689705-e5d5-4273-a678-ed401776e0bc None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:13.628029 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12689705-e5d5-4273-a678-ed401776e0bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:13.639943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe47e5f-9810-4a6b-bb50-0b71c7d48ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:13.640606 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe47e5f-9810-4a6b-bb50-0b71c7d48ca5 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:13.640895 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe47e5f-9810-4a6b-bb50-0b71c7d48ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:13.641171 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe47e5f-9810-4a6b-bb50-0b71c7d48ca5 None admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:13.641861 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe47e5f-9810-4a6b-bb50-0b71c7d48ca5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:13.642168 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe47e5f-9810-4a6b-bb50-0b71c7d48ca5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:13.642826 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12689705-e5d5-4273-a678-ed401776e0bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:13.643594 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe47e5f-9810-4a6b-bb50-0b71c7d48ca5 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-12689705-e5d5-4273-a678-ed401776e0bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:13.650968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:13.650968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:13.650968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:13.650968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:13.650968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:13.650968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:13.650968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:13.650968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:13.650968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:13.650968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:13.650968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:13.650968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:13.650968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:13.650968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:13.650968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:13.650968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:13.652143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:13.652143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:13.652143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:13.652143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:13.652143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:13.652143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:13.652143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:13.652143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:13.652143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:13.652143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:13.652143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:13.652143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:13.652143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 291/563] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Aug 26 00:35:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:13.652975 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e3872-c80e-4403-b28f-e02cff289e08 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:13.652975 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e3872-c80e-4403-b28f-e02cff289e08 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:13.653883 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e3872-c80e-4403-b28f-e02cff289e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:13.666587 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 273/564] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:35:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:13.666814 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:13.666921 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe47e5f-9810-4a6b-bb50-0b71c7d48ca5 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:13.682610 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-a05e3872-c80e-4403-b28f-e02cff289e08 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:13.684348 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a05e3872-c80e-4403-b28f-e02cff289e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:13.756418 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 292/565] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:35:13.757200 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e3872-c80e-4403-b28f-e02cff289e08 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:13.769692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe47e5f-9810-4a6b-bb50-0b71c7d48ca5 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:13.783421 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:13.803868 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea710a22-388d-4c64-9486-7593bcf0af0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:13.804514 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea710a22-388d-4c64-9486-7593bcf0af0c None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:13.804853 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea710a22-388d-4c64-9486-7593bcf0af0c None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:13.805137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea710a22-388d-4c64-9486-7593bcf0af0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:13.815488 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea710a22-388d-4c64-9486-7593bcf0af0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:13.817183 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea710a22-388d-4c64-9486-7593bcf0af0c None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:13.833054 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-ea710a22-388d-4c64-9486-7593bcf0af0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:13.833054 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:13.833054 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:13.833054 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:13.833054 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:13.833054 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:13.833054 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:13.833054 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:13.833054 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:13.833054 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:13.833054 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:13.833054 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:13.833054 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:13.833054 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:13.833054 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:13.833054 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:13.834016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:13.834016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:13.834016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:13.834016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:13.834016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:13.834016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:13.834016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:13.834016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:13.834016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:13.834016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:13.834016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:13.834016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:13.834016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:13.834016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:13.834016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:13.834016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:13.834016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:13.834916 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:13.834916 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:13.834916 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:13.838676 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 274/566] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:13.839522 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea710a22-388d-4c64-9486-7593bcf0af0c None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:13.875814 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e3872-c80e-4403-b28f-e02cff289e08 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:13.907358 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:13.913182 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979c43d-6a88-424e-ba4e-fb457ac85b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:13.913899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979c43d-6a88-424e-ba4e-fb457ac85b20 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:13.914177 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979c43d-6a88-424e-ba4e-fb457ac85b20 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:13.914446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979c43d-6a88-424e-ba4e-fb457ac85b20 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:13.915102 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0979c43d-6a88-424e-ba4e-fb457ac85b20 None admin] RBAC: Adding query filter params (name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:13.915444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0979c43d-6a88-424e-ba4e-fb457ac85b20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:13.916957 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0979c43d-6a88-424e-ba4e-fb457ac85b20 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:13.930270 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 293/567] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Aug 26 00:35:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:13.930270 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979c43d-6a88-424e-ba4e-fb457ac85b20 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:13.942709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea710a22-388d-4c64-9486-7593bcf0af0c None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:13.981360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06990b8-b789-4b04-af46-55bdd22af299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:13.981966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06990b8-b789-4b04-af46-55bdd22af299 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:13.982350 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06990b8-b789-4b04-af46-55bdd22af299 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:13.982526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06990b8-b789-4b04-af46-55bdd22af299 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:13.983215 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06990b8-b789-4b04-af46-55bdd22af299 None admin] RBAC: Adding query filter params (role.id=bce78c7e0a534135a820de7f3588aa4d, scope.project.id=fde7ac3ed472433ab3bba56e5e091510, user.id=2f700502671446c797a566f07398a9fc) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:13.983522 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06990b8-b789-4b04-af46-55bdd22af299 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:13.985620 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06990b8-b789-4b04-af46-55bdd22af299 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:14.000502 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 275/568] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:35:13 2022] GET /identity/v3/role_assignments?user.id=2f700502671446c797a566f07398a9fc&scope.project.id=fde7ac3ed472433ab3bba56e5e091510&role.id=bce78c7e0a534135a820de7f3588aa4d => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:14.000502 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06990b8-b789-4b04-af46-55bdd22af299 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:14.036384 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:14.061542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979c43d-6a88-424e-ba4e-fb457ac85b20 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:14.113799 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc0c56-9329-458c-9207-137a2b1f29d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:14.114443 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc0c56-9329-458c-9207-137a2b1f29d6 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:14.114724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc0c56-9329-458c-9207-137a2b1f29d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:14.114996 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc0c56-9329-458c-9207-137a2b1f29d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:14.126364 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fc0c56-9329-458c-9207-137a2b1f29d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:14.131480 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-01fc0c56-9329-458c-9207-137a2b1f29d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:14.131480 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:14.131480 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:14.131480 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:14.131480 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:14.131480 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:14.131480 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:14.131480 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:14.131480 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:14.131480 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:14.131480 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:14.131480 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:14.131480 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:14.131480 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:14.131480 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:14.131480 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:14.132635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:14.132635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:14.132635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:14.132635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:14.132635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:14.132635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:14.132635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:14.132635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:14.132635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:14.132635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:14.132635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:14.132635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:14.132635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:14.132635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:14.132635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:14.132635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:14.133498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:14.133498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:14.133498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:14.133498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:14.133498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:14.133498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:14.133498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:14.133498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:14.133498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:14.133498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:14.133498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:14.133498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:14.133498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 294/569] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Aug 26 00:35:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:14.134411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc0c56-9329-458c-9207-137a2b1f29d6 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:14.151283 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06990b8-b789-4b04-af46-55bdd22af299 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:14.188278 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c4d60-7ea4-4f7c-8b4a-1befc3074cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:14.188936 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c4d60-7ea4-4f7c-8b4a-1befc3074cf8 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:14.189211 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c4d60-7ea4-4f7c-8b4a-1befc3074cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:14.189479 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c4d60-7ea4-4f7c-8b4a-1befc3074cf8 None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:14.190119 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c4d60-7ea4-4f7c-8b4a-1befc3074cf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:14.190539 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c4d60-7ea4-4f7c-8b4a-1befc3074cf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:14.192008 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c4d60-7ea4-4f7c-8b4a-1befc3074cf8 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:14.197354 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:14.208831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 276/570] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:14.208831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c4d60-7ea4-4f7c-8b4a-1befc3074cf8 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:14.249808 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc0c56-9329-458c-9207-137a2b1f29d6 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:14.284722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cdcd62-fba6-4039-8355-5237b5d4bd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:14.285358 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdcd62-fba6-4039-8355-5237b5d4bd2c None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:14.285626 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdcd62-fba6-4039-8355-5237b5d4bd2c None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:14.285901 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdcd62-fba6-4039-8355-5237b5d4bd2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:14.297211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cdcd62-fba6-4039-8355-5237b5d4bd2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:14.298664 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cdcd62-fba6-4039-8355-5237b5d4bd2c None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:14.302084 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:14.312778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-d8cdcd62-fba6-4039-8355-5237b5d4bd2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:14.312778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:14.312778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:14.312778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:14.312778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:14.312778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:14.312778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:14.312778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:14.312778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:14.312778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:14.312778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:14.312778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:14.312778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:14.312778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:14.312778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:14.312778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:14.318444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:14.318444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:14.318444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:14.318444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:14.318444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:14.318444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:14.318444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:14.318444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:14.318444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:14.318444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:14.318444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:14.318444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:14.318444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:14.318444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:14.318444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:14.318444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:14.318444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:14.319513 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:14.319513 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:14.319513 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:14.319513 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 295/571] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:14.319796 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c4d60-7ea4-4f7c-8b4a-1befc3074cf8 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:14.319883 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cdcd62-fba6-4039-8355-5237b5d4bd2c None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:14.360686 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22e8f8-84b0-4833-abc3-96a30598e821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:14.361326 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22e8f8-84b0-4833-abc3-96a30598e821 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:14.361607 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22e8f8-84b0-4833-abc3-96a30598e821 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:14.361883 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22e8f8-84b0-4833-abc3-96a30598e821 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:14.376074 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22e8f8-84b0-4833-abc3-96a30598e821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:14.384260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-4f22e8f8-84b0-4833-abc3-96a30598e821 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:14.384260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:14.384260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:14.384260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:14.384260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:14.384260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:14.384260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:14.384260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:14.384260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:14.384260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:14.384260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:14.384260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:14.384260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:14.384260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:14.384260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:14.384260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:14.385482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:14.385482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:14.385482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:14.385482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:14.385482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:14.385482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:14.385482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:14.385482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:14.385482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:14.385482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:14.385482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:14.385482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:14.385482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:14.385482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:14.385482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:14.385482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:14.386331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:14.386331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:14.386331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:14.386331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:14.386331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:14.386331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:14.386331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:14.386331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:14.386331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:14.386331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:14.386331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:14.386331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:14.386331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 277/572] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Aug 26 00:35:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:14.387040 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22e8f8-84b0-4833-abc3-96a30598e821 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:14.435581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cdcd62-fba6-4039-8355-5237b5d4bd2c None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:14.443383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:14.471984 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39849165-b9d6-41fa-90d0-731e66bbcad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:14.472630 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39849165-b9d6-41fa-90d0-731e66bbcad8 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:14.472905 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39849165-b9d6-41fa-90d0-731e66bbcad8 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:14.473175 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39849165-b9d6-41fa-90d0-731e66bbcad8 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:14.473839 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39849165-b9d6-41fa-90d0-731e66bbcad8 None admin] RBAC: Adding query filter params (name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:14.474149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39849165-b9d6-41fa-90d0-731e66bbcad8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:14.475609 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39849165-b9d6-41fa-90d0-731e66bbcad8 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:14.484134 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 296/573] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Aug 26 00:35:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:14.485006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39849165-b9d6-41fa-90d0-731e66bbcad8 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:14.518627 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22e8f8-84b0-4833-abc3-96a30598e821 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:14.554806 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840e28d-7a33-4d6e-bc09-acdce305650d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:14.555717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840e28d-7a33-4d6e-bc09-acdce305650d None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:14.556148 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840e28d-7a33-4d6e-bc09-acdce305650d None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:14.556598 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840e28d-7a33-4d6e-bc09-acdce305650d None admin] PATH_INFO: `/v3/users/nova` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:14.565233 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840e28d-7a33-4d6e-bc09-acdce305650d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:14.566850 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840e28d-7a33-4d6e-bc09-acdce305650d None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:14.582372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-9840e28d-7a33-4d6e-bc09-acdce305650d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 00:35:14.582372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:14.582372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:14.582372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:14.582372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:14.582372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:14.582372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:14.582372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:14.582372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:14.582372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:14.582372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:14.582372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:14.582372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:35:14.582372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:35:14.582372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:35:14.582372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:35:14.583551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:14.583551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:14.583551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:14.583551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:14.583551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:14.583551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:14.583551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:14.583551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:14.583551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:14.583551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:14.583551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:14.583551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:14.583551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:14.583551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:14.583551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:14.583551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:14.583551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:14.583551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:14.584779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:14.584779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:14.584779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:35:14.584779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:35:14.584779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:35:14.584779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:35:14.584779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:35:14.584779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:35:14.584779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 00:35:14.584779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:14.587188 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 278/574] 10.222.0.59 () {56 vars in 1093 bytes} [Fri Aug 26 00:35:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:14.588213 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840e28d-7a33-4d6e-bc09-acdce305650d None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:14.600004 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39849165-b9d6-41fa-90d0-731e66bbcad8 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:14.602503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:14.639247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0848ae-4af1-418b-8ca6-fbaabc2ea3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:14.639784 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0848ae-4af1-418b-8ca6-fbaabc2ea3d1 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:14.640067 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0848ae-4af1-418b-8ca6-fbaabc2ea3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:14.640343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0848ae-4af1-418b-8ca6-fbaabc2ea3d1 None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:14.641026 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0848ae-4af1-418b-8ca6-fbaabc2ea3d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:14.641333 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0848ae-4af1-418b-8ca6-fbaabc2ea3d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:14.642782 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0848ae-4af1-418b-8ca6-fbaabc2ea3d1 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:14.659856 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 297/575] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:14.659856 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0848ae-4af1-418b-8ca6-fbaabc2ea3d1 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:14.687897 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:14.704871 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840e28d-7a33-4d6e-bc09-acdce305650d None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:14.745786 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e678b6b-c31c-4759-a694-3865854e9728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:14.746746 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e678b6b-c31c-4759-a694-3865854e9728 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:14.747230 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e678b6b-c31c-4759-a694-3865854e9728 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:14.747681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e678b6b-c31c-4759-a694-3865854e9728 None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:14.748598 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e678b6b-c31c-4759-a694-3865854e9728 None admin] RBAC: Adding query filter params (name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:14.749292 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e678b6b-c31c-4759-a694-3865854e9728 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:14.750915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e678b6b-c31c-4759-a694-3865854e9728 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:14.760517 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0848ae-4af1-418b-8ca6-fbaabc2ea3d1 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:14.764451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 279/576] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Aug 26 00:35:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:14.765206 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e678b6b-c31c-4759-a694-3865854e9728 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:14.795049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfc0fc-5412-4b0b-b6a7-72fd4b91145c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:14.795678 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfc0fc-5412-4b0b-b6a7-72fd4b91145c None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:14.795954 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfc0fc-5412-4b0b-b6a7-72fd4b91145c None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:14.796225 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfc0fc-5412-4b0b-b6a7-72fd4b91145c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:14.806705 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfc0fc-5412-4b0b-b6a7-72fd4b91145c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:14.808139 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfc0fc-5412-4b0b-b6a7-72fd4b91145c None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:14.822828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-c2cfc0fc-5412-4b0b-b6a7-72fd4b91145c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:14.822828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:14.822828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:14.822828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:14.822828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:14.822828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:14.822828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:14.822828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:14.822828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:14.822828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:14.822828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:14.822828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:14.822828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:14.822828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:14.822828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:14.822828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:14.823821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:14.823821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:14.823821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:14.823821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:14.823821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:14.823821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:14.823821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:14.823821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:14.823821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:14.823821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:14.823821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:14.823821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:14.823821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:14.823821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:14.823821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:14.823821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:14.823821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:14.825352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:14.825352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:14.825352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:14.825352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 298/577] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:14.825352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfc0fc-5412-4b0b-b6a7-72fd4b91145c None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:14.839347 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:14.875446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e678b6b-c31c-4759-a694-3865854e9728 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:14.910583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:14.911477 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ec26a-e961-4055-95d8-231e2ee330d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:14.912076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ec26a-e961-4055-95d8-231e2ee330d8 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:14.912406 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ec26a-e961-4055-95d8-231e2ee330d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:14.912720 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ec26a-e961-4055-95d8-231e2ee330d8 None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:14.913546 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ec26a-e961-4055-95d8-231e2ee330d8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:14.913841 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ec26a-e961-4055-95d8-231e2ee330d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:14.915266 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ec26a-e961-4055-95d8-231e2ee330d8 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:14.930689 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfc0fc-5412-4b0b-b6a7-72fd4b91145c None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:14.933643 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 280/578] 10.222.0.59 () {56 vars in 1133 bytes} [Fri Aug 26 00:35:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:14.934462 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ec26a-e961-4055-95d8-231e2ee330d8 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:14.972079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529e8de8-639f-4711-b140-28f06c3f0a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:14.972731 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e8de8-639f-4711-b140-28f06c3f0a70 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:14.973011 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e8de8-639f-4711-b140-28f06c3f0a70 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:14.973283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e8de8-639f-4711-b140-28f06c3f0a70 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:14.980682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529e8de8-639f-4711-b140-28f06c3f0a70 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:14.982117 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529e8de8-639f-4711-b140-28f06c3f0a70 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:14.995126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-529e8de8-639f-4711-b140-28f06c3f0a70 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 00:35:14.995126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:14.995126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:14.995126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:14.995126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:14.995126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:14.995126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:14.995126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:14.995126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:14.995126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:14.995126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:14.995126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:14.995126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:35:14.995126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:35:14.995126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:35:14.995126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:35:14.996086 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:14.996086 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:14.996086 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:14.996086 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:14.996086 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:14.996086 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:14.996086 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:14.996086 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:14.996086 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:14.996086 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:14.996086 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:14.996086 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:14.996086 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:14.996086 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:14.996086 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:14.996086 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:14.996086 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:14.996086 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:14.997089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:14.997089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:14.997089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:35:14.997089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:35:14.997089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:35:14.997089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:35:14.997089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:35:14.997089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:35:14.997089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 00:35:14.997089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:14.997089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 299/579] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:35:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:14.997089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529e8de8-639f-4711-b140-28f06c3f0a70 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:15.046767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ec26a-e961-4055-95d8-231e2ee330d8 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:15.101440 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:15.102695 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529e8de8-639f-4711-b140-28f06c3f0a70 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:15.110977 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1315b74-67ff-41d3-a4ce-4c9eb0d4e3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:15.118983 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1315b74-67ff-41d3-a4ce-4c9eb0d4e3b7 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:15.118983 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1315b74-67ff-41d3-a4ce-4c9eb0d4e3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:15.118983 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1315b74-67ff-41d3-a4ce-4c9eb0d4e3b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:15.122635 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1315b74-67ff-41d3-a4ce-4c9eb0d4e3b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:15.124072 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1315b74-67ff-41d3-a4ce-4c9eb0d4e3b7 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:15.137283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb99b1d-97af-4465-a122-75e852e7f98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:15.137886 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb99b1d-97af-4465-a122-75e852e7f98d None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:15.138160 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb99b1d-97af-4465-a122-75e852e7f98d None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:15.138430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb99b1d-97af-4465-a122-75e852e7f98d None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:15.138552 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-c1315b74-67ff-41d3-a4ce-4c9eb0d4e3b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:15.138552 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:15.138552 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:15.138552 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:15.138552 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:15.138552 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:15.138552 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:15.138552 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:15.138552 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:15.138552 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:15.138552 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:15.138552 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:15.138552 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:15.138552 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:15.138552 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:15.138552 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:15.139753 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:15.139753 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:15.139753 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:15.139753 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:15.139753 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:15.139753 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:15.139753 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:15.139753 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:15.139753 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:15.139753 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:15.139753 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:15.139753 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:15.139753 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:15.139753 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:15.139753 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:15.139753 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:15.139753 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:15.141558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:15.141558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:15.141558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:15.141734 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb99b1d-97af-4465-a122-75e852e7f98d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:15.141734 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb99b1d-97af-4465-a122-75e852e7f98d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:15.141734 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb99b1d-97af-4465-a122-75e852e7f98d None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:15.144805 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 281/580] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:15.145532 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1315b74-67ff-41d3-a4ce-4c9eb0d4e3b7 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:15.151708 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:15.155969 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 300/581] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:15.156729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb99b1d-97af-4465-a122-75e852e7f98d None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:15.247734 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1315b74-67ff-41d3-a4ce-4c9eb0d4e3b7 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:15.271010 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb99b1d-97af-4465-a122-75e852e7f98d None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:15.281160 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d5982-d140-4c4c-9319-67342289a8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:15.281784 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d5982-d140-4c4c-9319-67342289a8cb None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:15.282065 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d5982-d140-4c4c-9319-67342289a8cb None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:15.282336 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d5982-d140-4c4c-9319-67342289a8cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:15.292693 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779d5982-d140-4c4c-9319-67342289a8cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:15.294120 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779d5982-d140-4c4c-9319-67342289a8cb None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:15.308265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-779d5982-d140-4c4c-9319-67342289a8cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:15.308265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:15.308265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:15.308265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:15.308265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:15.308265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:15.308265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:15.308265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:15.308265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:15.308265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:15.308265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:15.308265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:15.308265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:15.308265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:15.308265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:15.308265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:15.309297 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:15.309297 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:15.309297 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:15.309297 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:15.309297 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:15.309297 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:15.309297 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:15.309297 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:15.309297 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:15.309297 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:15.309297 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:15.309297 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:15.309297 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:15.309297 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:15.309297 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:15.309297 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:15.309297 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:15.310215 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:15.310215 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:15.310215 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:15.310215 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 282/582] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:15.310215 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d5982-d140-4c4c-9319-67342289a8cb None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:15.310989 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f9605-412d-440f-abe2-229394ce111e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:15.312157 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f9605-412d-440f-abe2-229394ce111e None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:15.313188 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f9605-412d-440f-abe2-229394ce111e None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:15.313608 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f9605-412d-440f-abe2-229394ce111e None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:15.314686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f9605-412d-440f-abe2-229394ce111e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:15.315150 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f9605-412d-440f-abe2-229394ce111e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:15.316892 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f9605-412d-440f-abe2-229394ce111e None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:15.336998 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 301/583] 10.222.0.59 () {56 vars in 1137 bytes} [Fri Aug 26 00:35:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:15.337929 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f9605-412d-440f-abe2-229394ce111e None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:15.349221 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:15.415633 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:15.433364 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d5982-d140-4c4c-9319-67342289a8cb None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:15.459470 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f9605-412d-440f-abe2-229394ce111e None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:15.467933 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32503a45-92b8-42ee-841d-520eb534663a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:15.469064 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32503a45-92b8-42ee-841d-520eb534663a None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:15.469350 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32503a45-92b8-42ee-841d-520eb534663a None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:15.478083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32503a45-92b8-42ee-841d-520eb534663a None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:15.478083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32503a45-92b8-42ee-841d-520eb534663a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:15.478083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32503a45-92b8-42ee-841d-520eb534663a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:15.478083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32503a45-92b8-42ee-841d-520eb534663a None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:15.483762 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 283/584] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:15.485308 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32503a45-92b8-42ee-841d-520eb534663a None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:15.500457 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7251b-092f-4daf-956b-3e65c2384fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:15.501053 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7251b-092f-4daf-956b-3e65c2384fef None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:15.501335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7251b-092f-4daf-956b-3e65c2384fef None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:15.501607 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7251b-092f-4daf-956b-3e65c2384fef None admin] PATH_INFO: `/v3/users/glance` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:15.509334 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7251b-092f-4daf-956b-3e65c2384fef None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:15.510782 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7251b-092f-4daf-956b-3e65c2384fef None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:15.522580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-dfb7251b-092f-4daf-956b-3e65c2384fef None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 26 00:35:15.522580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:15.522580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:15.522580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:15.522580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:15.522580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:15.522580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:15.522580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:15.522580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:15.522580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:15.522580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:15.522580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:15.522580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:35:15.522580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:35:15.522580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:35:15.522580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:35:15.523554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:15.523554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:15.523554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:15.523554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:15.523554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:15.523554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:15.523554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:15.523554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:15.523554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:15.523554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:15.523554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:15.523554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:15.523554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:15.523554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:15.523554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:15.523554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:15.523554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:15.523554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:15.524533 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:15.524533 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:15.524533 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:35:15.524533 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:35:15.524533 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:35:15.524533 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:35:15.524533 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:35:15.524533 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:35:15.524533 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 26 00:35:15.524533 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:15.524533 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 302/585] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:35:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:15.529240 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7251b-092f-4daf-956b-3e65c2384fef None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:15.588201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:15.598355 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32503a45-92b8-42ee-841d-520eb534663a None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:15.634619 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef4533-6252-463f-ad6f-80d0c6ae86cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:15.635227 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef4533-6252-463f-ad6f-80d0c6ae86cc None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:15.635606 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef4533-6252-463f-ad6f-80d0c6ae86cc None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:15.635786 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef4533-6252-463f-ad6f-80d0c6ae86cc None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:15.636427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef4533-6252-463f-ad6f-80d0c6ae86cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:15.636765 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef4533-6252-463f-ad6f-80d0c6ae86cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:15.638133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef4533-6252-463f-ad6f-80d0c6ae86cc None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:15.639724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7251b-092f-4daf-956b-3e65c2384fef None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:15.654018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 284/586] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:15.654018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef4533-6252-463f-ad6f-80d0c6ae86cc None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:15.663010 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:15.680560 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa69b2-89ca-473c-81b6-6a546d152f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:15.681304 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa69b2-89ca-473c-81b6-6a546d152f82 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:15.681567 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa69b2-89ca-473c-81b6-6a546d152f82 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:15.691151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa69b2-89ca-473c-81b6-6a546d152f82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:15.693439 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aa69b2-89ca-473c-81b6-6a546d152f82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:15.694921 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aa69b2-89ca-473c-81b6-6a546d152f82 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:15.710029 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-46aa69b2-89ca-473c-81b6-6a546d152f82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:15.710029 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:15.710029 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:15.710029 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:15.710029 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:15.710029 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:15.710029 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:15.710029 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:15.710029 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:15.710029 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:15.710029 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:15.710029 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:15.710029 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:15.710029 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:15.710029 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:15.710029 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:15.716413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:15.716413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:15.716413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:15.716413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:15.716413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:15.716413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:15.716413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:15.716413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:15.716413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:15.716413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:15.716413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:15.716413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:15.716413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:15.716413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:15.716413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:15.716413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:15.716413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:15.717594 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:15.717594 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:15.717594 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:15.717594 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 303/587] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:15.717594 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa69b2-89ca-473c-81b6-6a546d152f82 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:15.770662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef4533-6252-463f-ad6f-80d0c6ae86cc None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:15.809118 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e65ba2-3512-44f7-b964-6cd1acd1066a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:15.809854 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e65ba2-3512-44f7-b964-6cd1acd1066a None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:15.810196 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e65ba2-3512-44f7-b964-6cd1acd1066a None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:15.810538 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e65ba2-3512-44f7-b964-6cd1acd1066a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:15.821685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e65ba2-3512-44f7-b964-6cd1acd1066a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:15.823227 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e65ba2-3512-44f7-b964-6cd1acd1066a None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:15.831079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa69b2-89ca-473c-81b6-6a546d152f82 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:15.834924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-85e65ba2-3512-44f7-b964-6cd1acd1066a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 00:35:15.834924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:15.834924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:15.834924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:15.834924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:15.834924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:15.834924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:15.834924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:15.834924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:15.834924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:15.834924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:15.834924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:15.834924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:35:15.834924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:35:15.834924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:35:15.834924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:35:15.835899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:15.835899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:15.835899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:15.835899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:15.835899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:15.835899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:15.835899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:15.835899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:15.835899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:15.835899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:15.835899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:15.835899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:15.835899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:15.835899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:15.835899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:15.835899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:15.835899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:15.835899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:15.837365 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:15.837365 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:15.837365 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:35:15.837365 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:35:15.837365 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:35:15.837365 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:35:15.837365 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:35:15.837365 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:35:15.837365 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 00:35:15.837365 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:15.837365 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 285/588] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Aug 26 00:35:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:15.837365 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e65ba2-3512-44f7-b964-6cd1acd1066a None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:15.853257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:15.871824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee3dc1a-3276-4cfc-b220-4f9e740baea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:15.872431 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3dc1a-3276-4cfc-b220-4f9e740baea6 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:15.872781 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3dc1a-3276-4cfc-b220-4f9e740baea6 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:15.873060 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3dc1a-3276-4cfc-b220-4f9e740baea6 None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:15.873792 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee3dc1a-3276-4cfc-b220-4f9e740baea6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:15.874077 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee3dc1a-3276-4cfc-b220-4f9e740baea6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:15.875545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee3dc1a-3276-4cfc-b220-4f9e740baea6 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:15.894923 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 304/589] 10.222.0.59 () {56 vars in 1137 bytes} [Fri Aug 26 00:35:15 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:15.896099 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee3dc1a-3276-4cfc-b220-4f9e740baea6 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:15.927190 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:15.950136 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e65ba2-3512-44f7-b964-6cd1acd1066a None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:15.986075 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1bd3cf-8187-4c67-9503-428960a20711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:15.986709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bd3cf-8187-4c67-9503-428960a20711 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:15.986989 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bd3cf-8187-4c67-9503-428960a20711 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:15.987262 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bd3cf-8187-4c67-9503-428960a20711 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:15.993940 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1bd3cf-8187-4c67-9503-428960a20711 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:15.995376 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1bd3cf-8187-4c67-9503-428960a20711 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:16.004003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee3dc1a-3276-4cfc-b220-4f9e740baea6 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:16.006856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-cf1bd3cf-8187-4c67-9503-428960a20711 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:16.006856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:16.006856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:16.006856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:16.006856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:16.006856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:16.006856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:16.006856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:16.006856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:16.006856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:16.006856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:16.006856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:16.006856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:35:16.006856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:35:16.006856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:35:16.007726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:35:16.007726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:16.007726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:16.007726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:16.007726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:16.007726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:16.007726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:16.007726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:16.007726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:16.007726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:16.007726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:16.007726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:16.007726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:16.007726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:16.007726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:16.007726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:16.007726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:16.008794 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:35:16.008794 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:35:16.008794 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:35:16.008794 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:35:16.008794 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:35:16.008794 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:35:16.008794 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:16.008794 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:16.008794 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 286/590] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:16.008794 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1bd3cf-8187-4c67-9503-428960a20711 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:16.042021 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af46a3a0-bbfe-4eea-acd3-a6abc91be7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:16.042547 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af46a3a0-bbfe-4eea-acd3-a6abc91be7ac None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:16.042827 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af46a3a0-bbfe-4eea-acd3-a6abc91be7ac None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:16.043098 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af46a3a0-bbfe-4eea-acd3-a6abc91be7ac None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:16.043745 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af46a3a0-bbfe-4eea-acd3-a6abc91be7ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:16.044042 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af46a3a0-bbfe-4eea-acd3-a6abc91be7ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:16.045434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af46a3a0-bbfe-4eea-acd3-a6abc91be7ac None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:16.061393 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 305/591] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:16.061393 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af46a3a0-bbfe-4eea-acd3-a6abc91be7ac None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:16.094469 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:16.123529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1bd3cf-8187-4c67-9503-428960a20711 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:16.146740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:16.158726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af46a3a0-bbfe-4eea-acd3-a6abc91be7ac None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:16.159354 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb1d3e9-ec25-47b1-9661-7344cae032fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:16.159961 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1d3e9-ec25-47b1-9661-7344cae032fa None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:16.160247 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1d3e9-ec25-47b1-9661-7344cae032fa None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:16.160554 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1d3e9-ec25-47b1-9661-7344cae032fa None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:16.161285 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb1d3e9-ec25-47b1-9661-7344cae032fa None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:16.161644 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb1d3e9-ec25-47b1-9661-7344cae032fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:16.163063 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb1d3e9-ec25-47b1-9661-7344cae032fa None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:16.181048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 287/592] 10.222.0.59 () {56 vars in 1139 bytes} [Fri Aug 26 00:35:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:16.183039 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602d9c5-1beb-47ba-802f-40f059c4c838 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:16.183554 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602d9c5-1beb-47ba-802f-40f059c4c838 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:16.184025 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602d9c5-1beb-47ba-802f-40f059c4c838 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:16.188514 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 288/593] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:16.188514 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602d9c5-1beb-47ba-802f-40f059c4c838 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:16.197303 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901cb8b-50c8-492b-93de-2f5a8c67e9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:16.197929 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901cb8b-50c8-492b-93de-2f5a8c67e9f1 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:16.198204 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901cb8b-50c8-492b-93de-2f5a8c67e9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:16.198481 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901cb8b-50c8-492b-93de-2f5a8c67e9f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:16.209797 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901cb8b-50c8-492b-93de-2f5a8c67e9f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:16.211251 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901cb8b-50c8-492b-93de-2f5a8c67e9f1 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:16.225539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-b901cb8b-50c8-492b-93de-2f5a8c67e9f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:16.225539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:16.225539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:16.225539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:16.225539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:16.225539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:16.225539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:16.225539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:16.225539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:16.225539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:16.225539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:16.225539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:16.225539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:16.225539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:16.225539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:16.225539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:16.226740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:16.226740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:16.226740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:16.226740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:16.226740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:16.226740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:16.226740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:16.226740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:16.226740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:16.226740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:16.226740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:16.226740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:16.226740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:16.226740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:16.226740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:16.226740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:16.226740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:16.227940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:16.227940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:16.227940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:16.231245 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 306/594] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:16.232029 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901cb8b-50c8-492b-93de-2f5a8c67e9f1 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:16.289727 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602d9c5-1beb-47ba-802f-40f059c4c838 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:16.317989 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:16.323480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc64dc-b3f6-4017-b02d-e4fa98337a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:16.324087 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc64dc-b3f6-4017-b02d-e4fa98337a49 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:16.324363 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc64dc-b3f6-4017-b02d-e4fa98337a49 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:16.324669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc64dc-b3f6-4017-b02d-e4fa98337a49 None admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:16.325354 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc64dc-b3f6-4017-b02d-e4fa98337a49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:16.325656 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc64dc-b3f6-4017-b02d-e4fa98337a49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:16.327071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc64dc-b3f6-4017-b02d-e4fa98337a49 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:16.336417 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901cb8b-50c8-492b-93de-2f5a8c67e9f1 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:16.339517 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 289/595] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:35:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:16.340391 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc64dc-b3f6-4017-b02d-e4fa98337a49 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:16.372558 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f27455-b59e-4b0c-9750-707ee09921df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:16.373194 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f27455-b59e-4b0c-9750-707ee09921df None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:16.373478 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f27455-b59e-4b0c-9750-707ee09921df None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:16.373754 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f27455-b59e-4b0c-9750-707ee09921df None admin] PATH_INFO: `/v3/projects/service` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:16.381449 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f27455-b59e-4b0c-9750-707ee09921df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:16.383784 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f27455-b59e-4b0c-9750-707ee09921df None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:16.386380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:16.396088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-90f27455-b59e-4b0c-9750-707ee09921df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:16.396088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:16.396088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:16.396088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:16.396088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:16.396088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:16.396088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:16.396088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:16.396088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:16.396088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:16.396088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:16.396088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:16.396088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:35:16.396088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:35:16.396088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:35:16.397038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:35:16.397038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:16.397038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:16.397038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:16.397038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:16.397038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:16.397038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:16.397038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:16.397038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:16.397038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:16.397038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:16.397038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:16.397038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:16.397038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:16.397038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:16.397038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:16.397038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:16.397939 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:35:16.397939 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:35:16.397939 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:35:16.397939 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:35:16.397939 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:35:16.397939 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:35:16.397939 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:16.397939 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:16.397939 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 307/596] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:16.398431 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07859ff8-682e-454f-b462-6bc844f42b1e None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:16.398702 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07859ff8-682e-454f-b462-6bc844f42b1e None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:16.399017 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07859ff8-682e-454f-b462-6bc844f42b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:16.417742 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-07859ff8-682e-454f-b462-6bc844f42b1e None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:16.418962 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-07859ff8-682e-454f-b462-6bc844f42b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:16.447456 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc64dc-b3f6-4017-b02d-e4fa98337a49 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:16.482336 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11dca9-6bc3-4ae8-b139-6574e949e05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:16.482977 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11dca9-6bc3-4ae8-b139-6574e949e05f None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:16.483260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11dca9-6bc3-4ae8-b139-6574e949e05f None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:16.483535 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11dca9-6bc3-4ae8-b139-6574e949e05f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:16.492659 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 308/597] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:35:16.493511 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07859ff8-682e-454f-b462-6bc844f42b1e None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:16.494827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11dca9-6bc3-4ae8-b139-6574e949e05f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:16.497105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11dca9-6bc3-4ae8-b139-6574e949e05f None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:16.513286 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-7d11dca9-6bc3-4ae8-b139-6574e949e05f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:16.513286 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:16.513286 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:16.513286 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:16.513286 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:16.513286 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:16.513286 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:16.513286 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:16.513286 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:16.513286 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:16.513286 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:16.513286 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:16.513286 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:16.513286 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:16.513286 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:16.513286 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:16.514289 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:16.514289 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:16.514289 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:16.514289 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:16.514289 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:16.514289 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:16.514289 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:16.514289 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:16.514289 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:16.514289 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:16.514289 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:16.514289 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:16.514289 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:16.514289 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:16.514289 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:16.514289 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:16.514289 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:16.515246 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:16.515246 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:16.515246 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:16.515246 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 290/598] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:16.515246 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11dca9-6bc3-4ae8-b139-6574e949e05f None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:16.562076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:16.600273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07859ff8-682e-454f-b462-6bc844f42b1e None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:16.624264 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11dca9-6bc3-4ae8-b139-6574e949e05f None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:16.629939 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:16.636034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62861b31-a3fb-48d0-af01-f6d4852d1906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:16.636829 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62861b31-a3fb-48d0-af01-f6d4852d1906 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:16.637112 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62861b31-a3fb-48d0-af01-f6d4852d1906 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:16.637479 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62861b31-a3fb-48d0-af01-f6d4852d1906 None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:16.638029 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62861b31-a3fb-48d0-af01-f6d4852d1906 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:16.641695 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62861b31-a3fb-48d0-af01-f6d4852d1906 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:16.641695 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62861b31-a3fb-48d0-af01-f6d4852d1906 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:16.655726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 309/599] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:16.655726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62861b31-a3fb-48d0-af01-f6d4852d1906 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:16.665731 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4ba90-7626-4bc2-bd96-50815f569c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:16.666232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4ba90-7626-4bc2-bd96-50815f569c9c None admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:16.666509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4ba90-7626-4bc2-bd96-50815f569c9c None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:16.666865 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4ba90-7626-4bc2-bd96-50815f569c9c None admin] PATH_INFO: `/v3/projects/fde7ac3ed472433ab3bba56e5e091510/users/0602593263ca4b05a657d5b7697f932f/roles/bce78c7e0a534135a820de7f3588aa4d` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:16.683735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4ba90-7626-4bc2-bd96-50815f569c9c None admin] RBAC: Authorizing `identity:create_grant(project_id=fde7ac3ed472433ab3bba56e5e091510, user_id=0602593263ca4b05a657d5b7697f932f, role_id=bce78c7e0a534135a820de7f3588aa4d)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:16.686219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4ba90-7626-4bc2-bd96-50815f569c9c None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:16.753963 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62861b31-a3fb-48d0-af01-f6d4852d1906 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:16.776502 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 291/600] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Aug 26 00:35:16 2022] PUT /identity/v3/projects/fde7ac3ed472433ab3bba56e5e091510/users/0602593263ca4b05a657d5b7697f932f/roles/bce78c7e0a534135a820de7f3588aa4d => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:35:16.776502 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4ba90-7626-4bc2-bd96-50815f569c9c None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:16.793987 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b8c8d-7461-46a5-9690-7728bf232cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:16.794777 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b8c8d-7461-46a5-9690-7728bf232cfd None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:16.795060 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b8c8d-7461-46a5-9690-7728bf232cfd None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:16.795330 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b8c8d-7461-46a5-9690-7728bf232cfd None admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:16.796115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0b8c8d-7461-46a5-9690-7728bf232cfd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:16.796384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0b8c8d-7461-46a5-9690-7728bf232cfd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:16.798884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0b8c8d-7461-46a5-9690-7728bf232cfd None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:16.818134 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 310/601] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:35:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:16.818134 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b8c8d-7461-46a5-9690-7728bf232cfd None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:16.874230 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:16.885700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:16.895556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4ba90-7626-4bc2-bd96-50815f569c9c None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:16.936570 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26cf1c-96b5-443d-be0d-67b6d37a0538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:16.937477 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26cf1c-96b5-443d-be0d-67b6d37a0538 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:16.937907 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26cf1c-96b5-443d-be0d-67b6d37a0538 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:16.938021 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b8c8d-7461-46a5-9690-7728bf232cfd None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:16.938422 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26cf1c-96b5-443d-be0d-67b6d37a0538 None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:16.939210 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de26cf1c-96b5-443d-be0d-67b6d37a0538 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:16.939668 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de26cf1c-96b5-443d-be0d-67b6d37a0538 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:16.941210 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de26cf1c-96b5-443d-be0d-67b6d37a0538 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:16.953218 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 292/602] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:16.958469 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26cf1c-96b5-443d-be0d-67b6d37a0538 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:16.975407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2cced-8f71-44ee-9b1a-9b4ce20aae30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:16.976047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2cced-8f71-44ee-9b1a-9b4ce20aae30 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:16.976330 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2cced-8f71-44ee-9b1a-9b4ce20aae30 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:16.976691 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2cced-8f71-44ee-9b1a-9b4ce20aae30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:16.983582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba2cced-8f71-44ee-9b1a-9b4ce20aae30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:16.985602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba2cced-8f71-44ee-9b1a-9b4ce20aae30 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:16.996884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-9ba2cced-8f71-44ee-9b1a-9b4ce20aae30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:16.996884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:16.996884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:16.996884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:16.996884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:16.996884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:16.996884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:16.996884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:16.996884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:16.996884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:16.996884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:16.996884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:16.996884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:35:16.996884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:35:16.996884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:35:16.997801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:35:16.997801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:16.997801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:16.997801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:16.997801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:16.997801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:16.997801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:16.997801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:16.997801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:16.997801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:16.997801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:16.997801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:16.997801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:16.997801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:16.997801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:16.997801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:16.997801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:16.998711 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:35:16.998711 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:35:16.998711 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:35:16.998711 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:35:16.998711 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:35:16.998711 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:35:16.998711 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:16.998711 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:16.998711 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 311/603] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:16.998711 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2cced-8f71-44ee-9b1a-9b4ce20aae30 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:17.069295 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26cf1c-96b5-443d-be0d-67b6d37a0538 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:17.104785 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c5a16-8b86-44ba-bec6-26d86a54b686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:17.105422 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c5a16-8b86-44ba-bec6-26d86a54b686 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:17.105702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c5a16-8b86-44ba-bec6-26d86a54b686 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:17.105974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c5a16-8b86-44ba-bec6-26d86a54b686 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:17.106469 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2cced-8f71-44ee-9b1a-9b4ce20aae30 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:17.116648 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c5a16-8b86-44ba-bec6-26d86a54b686 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:17.119256 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:17.122487 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-990c5a16-8b86-44ba-bec6-26d86a54b686 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:17.122487 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:17.122487 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:17.122487 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:17.122487 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:17.122487 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:17.122487 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:17.122487 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:17.122487 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:17.122487 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:17.122487 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:17.122487 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:17.122487 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:17.122487 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:17.122487 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:17.122487 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:17.123427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:17.123427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:17.123427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:17.123427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:17.123427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:17.123427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:17.123427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:17.123427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:17.123427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:17.123427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:17.123427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:17.123427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:17.123427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:17.123427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:17.123427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:17.123427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:17.124357 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:17.124357 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:17.124357 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:17.124357 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:17.124357 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:17.124357 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:17.124357 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:17.124357 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:17.124357 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:17.124357 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:17.124357 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:17.124357 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:17.124357 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 293/604] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Aug 26 00:35:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:17.124357 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c5a16-8b86-44ba-bec6-26d86a54b686 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:17.139891 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:17.148868 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96d0dc-d479-40f8-8303-858a155a186d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:17.149537 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96d0dc-d479-40f8-8303-858a155a186d None admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:17.149822 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96d0dc-d479-40f8-8303-858a155a186d None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:17.150102 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96d0dc-d479-40f8-8303-858a155a186d None admin] PATH_INFO: `/v3/projects/fde7ac3ed472433ab3bba56e5e091510/users/c88202614775418ba94f478f03431a9b/roles/bce78c7e0a534135a820de7f3588aa4d` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:17.166172 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa96d0dc-d479-40f8-8303-858a155a186d None admin] RBAC: Authorizing `identity:create_grant(project_id=fde7ac3ed472433ab3bba56e5e091510, user_id=c88202614775418ba94f478f03431a9b, role_id=bce78c7e0a534135a820de7f3588aa4d)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:17.168143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa96d0dc-d479-40f8-8303-858a155a186d None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:17.227453 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c5a16-8b86-44ba-bec6-26d86a54b686 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:17.260552 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 312/605] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Aug 26 00:35:16 2022] PUT /identity/v3/projects/fde7ac3ed472433ab3bba56e5e091510/users/c88202614775418ba94f478f03431a9b/roles/bce78c7e0a534135a820de7f3588aa4d => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:35:17.261619 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96d0dc-d479-40f8-8303-858a155a186d None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:17.270768 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80fa13-bfd7-44a6-85fc-8387df36bc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:17.271775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80fa13-bfd7-44a6-85fc-8387df36bc0a None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:17.272264 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80fa13-bfd7-44a6-85fc-8387df36bc0a None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:17.272731 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80fa13-bfd7-44a6-85fc-8387df36bc0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:17.282214 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80fa13-bfd7-44a6-85fc-8387df36bc0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:17.283956 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80fa13-bfd7-44a6-85fc-8387df36bc0a None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:17.297332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-dd80fa13-bfd7-44a6-85fc-8387df36bc0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:17.297332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:17.297332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:17.297332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:17.297332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:17.297332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:17.297332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:17.297332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:17.297332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:17.297332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:17.297332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:17.297332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:17.297332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:35:17.297332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:35:17.297332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:35:17.298510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:35:17.298510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:17.298510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:17.298510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:17.298510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:17.298510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:17.298510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:17.298510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:17.298510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:17.298510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:17.298510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:17.298510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:17.298510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:17.298510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:17.298510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:17.298510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:17.298510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:17.299640 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:35:17.299640 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:35:17.299640 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:35:17.299640 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:35:17.299640 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:35:17.299640 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:35:17.299640 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:17.299640 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:17.301732 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 294/606] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:17.302713 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80fa13-bfd7-44a6-85fc-8387df36bc0a None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:17.397208 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:17.418189 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96d0dc-d479-40f8-8303-858a155a186d None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:17.427451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80fa13-bfd7-44a6-85fc-8387df36bc0a None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:17.464512 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b455c8e-a703-44bc-a747-e30f99b3db45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:17.465312 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b455c8e-a703-44bc-a747-e30f99b3db45 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:17.465695 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b455c8e-a703-44bc-a747-e30f99b3db45 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:17.466071 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b455c8e-a703-44bc-a747-e30f99b3db45 None admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:17.466881 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b455c8e-a703-44bc-a747-e30f99b3db45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:17.467308 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b455c8e-a703-44bc-a747-e30f99b3db45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:17.468942 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b455c8e-a703-44bc-a747-e30f99b3db45 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:17.480305 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633bc5f-c231-4352-826b-a4fca9b05ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:17.481140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633bc5f-c231-4352-826b-a4fca9b05ab5 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:17.481361 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 313/607] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:35:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:17.481773 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633bc5f-c231-4352-826b-a4fca9b05ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:17.482261 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633bc5f-c231-4352-826b-a4fca9b05ab5 None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:17.482460 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b455c8e-a703-44bc-a747-e30f99b3db45 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:17.483331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633bc5f-c231-4352-826b-a4fca9b05ab5 None admin] RBAC: Adding query filter params (name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:17.483783 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633bc5f-c231-4352-826b-a4fca9b05ab5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:17.486167 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633bc5f-c231-4352-826b-a4fca9b05ab5 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:17.497166 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 295/608] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Aug 26 00:35:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:17.498265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633bc5f-c231-4352-826b-a4fca9b05ab5 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:17.519714 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:17.612393 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b455c8e-a703-44bc-a747-e30f99b3db45 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:17.620631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633bc5f-c231-4352-826b-a4fca9b05ab5 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:17.657189 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc50f7-6554-4236-b12c-ead9fc88ece4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:17.657922 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc50f7-6554-4236-b12c-ead9fc88ece4 None admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:17.658324 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc50f7-6554-4236-b12c-ead9fc88ece4 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:17.658701 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc50f7-6554-4236-b12c-ead9fc88ece4 None admin] PATH_INFO: `/v3/projects/fde7ac3ed472433ab3bba56e5e091510/users/abed7ebb0aae4e4693ce08e51a149312/roles/bce78c7e0a534135a820de7f3588aa4d` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:17.659964 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2bc9be-dff5-403d-b8f5-f55c284c988e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:17.660826 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bc9be-dff5-403d-b8f5-f55c284c988e None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:17.661265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bc9be-dff5-403d-b8f5-f55c284c988e None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:17.661885 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:17.662275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bc9be-dff5-403d-b8f5-f55c284c988e None admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:17.664363 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2bc9be-dff5-403d-b8f5-f55c284c988e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:17.665040 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2bc9be-dff5-403d-b8f5-f55c284c988e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:17.666635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2bc9be-dff5-403d-b8f5-f55c284c988e None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:17.676047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc50f7-6554-4236-b12c-ead9fc88ece4 None admin] RBAC: Authorizing `identity:create_grant(project_id=fde7ac3ed472433ab3bba56e5e091510, user_id=abed7ebb0aae4e4693ce08e51a149312, role_id=bce78c7e0a534135a820de7f3588aa4d)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:17.677713 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc50f7-6554-4236-b12c-ead9fc88ece4 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:17.682679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 314/609] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:35:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:17.683726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2bc9be-dff5-403d-b8f5-f55c284c988e None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:17.771156 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:17.791353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2bc9be-dff5-403d-b8f5-f55c284c988e None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:17.800882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 296/610] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Aug 26 00:35:17 2022] PUT /identity/v3/projects/fde7ac3ed472433ab3bba56e5e091510/users/abed7ebb0aae4e4693ce08e51a149312/roles/bce78c7e0a534135a820de7f3588aa4d => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:35:17.802066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc50f7-6554-4236-b12c-ead9fc88ece4 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:17.830591 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a6b0f3-e263-49ee-8ee9-8255149de5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:17.831331 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6b0f3-e263-49ee-8ee9-8255149de5a7 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:17.831681 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6b0f3-e263-49ee-8ee9-8255149de5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:17.832022 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6b0f3-e263-49ee-8ee9-8255149de5a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:17.855560 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a6b0f3-e263-49ee-8ee9-8255149de5a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:17.857326 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a6b0f3-e263-49ee-8ee9-8255149de5a7 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:17.889502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-24a6b0f3-e263-49ee-8ee9-8255149de5a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:17.889502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:17.889502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:17.889502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:17.889502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:17.889502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:17.889502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:17.889502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:17.889502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:17.889502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:17.889502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:17.889502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:17.889502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:17.889502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:17.889502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:17.889502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:17.890772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:17.890772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:17.890772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:17.890772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:17.890772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:17.890772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:17.890772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:17.890772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:17.890772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:17.890772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:17.890772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:17.890772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:17.890772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:17.890772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:17.890772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:17.890772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:17.890772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:17.891694 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:17.891694 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:17.891694 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:17.897683 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 315/611] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:17.903748 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a6b0f3-e263-49ee-8ee9-8255149de5a7 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:17.952853 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc50f7-6554-4236-b12c-ead9fc88ece4 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:18.013842 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ed266-829a-4e33-83f4-44dec2542407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:18.014970 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ed266-829a-4e33-83f4-44dec2542407 None admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:18.015907 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ed266-829a-4e33-83f4-44dec2542407 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:18.015907 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ed266-829a-4e33-83f4-44dec2542407 None admin] PATH_INFO: `/v3/projects/fde7ac3ed472433ab3bba56e5e091510/users/4698a49ed4a04f58a488359a2742e5da/roles/bce78c7e0a534135a820de7f3588aa4d` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:18.038174 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ed266-829a-4e33-83f4-44dec2542407 None admin] RBAC: Authorizing `identity:create_grant(project_id=fde7ac3ed472433ab3bba56e5e091510, user_id=4698a49ed4a04f58a488359a2742e5da, role_id=bce78c7e0a534135a820de7f3588aa4d)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:18.039610 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ed266-829a-4e33-83f4-44dec2542407 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:18.072395 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a6b0f3-e263-49ee-8ee9-8255149de5a7 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:18.105163 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:18.110363 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d419e36-b684-4ba9-9309-665f750a17d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:18.110990 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d419e36-b684-4ba9-9309-665f750a17d0 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:18.111260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d419e36-b684-4ba9-9309-665f750a17d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:18.111533 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d419e36-b684-4ba9-9309-665f750a17d0 None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:18.112181 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d419e36-b684-4ba9-9309-665f750a17d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:18.112500 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d419e36-b684-4ba9-9309-665f750a17d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:18.113892 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d419e36-b684-4ba9-9309-665f750a17d0 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:18.126654 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 316/612] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:35:18.132812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d419e36-b684-4ba9-9309-665f750a17d0 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:18.136360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 297/613] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Aug 26 00:35:17 2022] PUT /identity/v3/projects/fde7ac3ed472433ab3bba56e5e091510/users/4698a49ed4a04f58a488359a2742e5da/roles/bce78c7e0a534135a820de7f3588aa4d => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:35:18.252185 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d419e36-b684-4ba9-9309-665f750a17d0 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:18.291800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340df507-5f7f-477b-98d3-bc0053dc73a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:18.292529 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340df507-5f7f-477b-98d3-bc0053dc73a2 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:18.292817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340df507-5f7f-477b-98d3-bc0053dc73a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:18.293148 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340df507-5f7f-477b-98d3-bc0053dc73a2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:18.301440 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340df507-5f7f-477b-98d3-bc0053dc73a2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:18.302934 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340df507-5f7f-477b-98d3-bc0053dc73a2 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:18.316343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-340df507-5f7f-477b-98d3-bc0053dc73a2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 00:35:18.316343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:18.316343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:18.316343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:18.316343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:18.316343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:18.316343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:18.316343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:18.316343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:18.316343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:18.316343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:18.316343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:18.316343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:35:18.316343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:35:18.316343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:35:18.316343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:35:18.317390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:18.317390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:18.317390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:18.317390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:18.317390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:18.317390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:18.317390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:18.317390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:18.317390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:18.317390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:18.317390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:18.317390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:18.317390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:18.317390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:18.317390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:18.317390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:18.317390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:18.317390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:18.319386 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:18.319386 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:18.319386 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:35:18.319386 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:35:18.319386 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:35:18.319386 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:35:18.319386 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:35:18.319386 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:35:18.319386 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 00:35:18.319386 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:18.319386 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 317/614] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:35:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:18.323291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ed266-829a-4e33-83f4-44dec2542407 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:18.366320 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:18.443150 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ed266-829a-4e33-83f4-44dec2542407 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:18.498315 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e85680-ea66-4a0e-a61a-bd4c787bdac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:18.499218 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e85680-ea66-4a0e-a61a-bd4c787bdac9 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:18.499695 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e85680-ea66-4a0e-a61a-bd4c787bdac9 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:18.500123 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e85680-ea66-4a0e-a61a-bd4c787bdac9 None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:18.501031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e85680-ea66-4a0e-a61a-bd4c787bdac9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:18.501484 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e85680-ea66-4a0e-a61a-bd4c787bdac9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:18.503116 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e85680-ea66-4a0e-a61a-bd4c787bdac9 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:18.527863 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 298/615] 10.222.0.59 () {56 vars in 1135 bytes} [Fri Aug 26 00:35:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:18.534399 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340df507-5f7f-477b-98d3-bc0053dc73a2 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:18.617737 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:18.676303 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340df507-5f7f-477b-98d3-bc0053dc73a2 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:18.719763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14faadcf-bebc-4c47-b759-a9dc43708ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:18.720655 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14faadcf-bebc-4c47-b759-a9dc43708ce7 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:18.720939 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14faadcf-bebc-4c47-b759-a9dc43708ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:18.721215 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14faadcf-bebc-4c47-b759-a9dc43708ce7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:18.737268 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14faadcf-bebc-4c47-b759-a9dc43708ce7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:18.738736 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14faadcf-bebc-4c47-b759-a9dc43708ce7 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:18.757239 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-14faadcf-bebc-4c47-b759-a9dc43708ce7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:18.757239 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:18.757239 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:18.757239 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:18.757239 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:18.757239 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:18.757239 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:18.757239 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:18.757239 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:18.757239 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:18.757239 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:18.757239 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:18.757239 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:18.757239 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:18.757239 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:18.757239 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:18.758234 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:18.758234 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:18.758234 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:18.758234 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:18.758234 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:18.758234 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:18.758234 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:18.758234 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:18.758234 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:18.758234 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:18.758234 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:18.758234 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:18.758234 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:18.758234 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:18.758234 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:18.758234 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:18.758234 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:18.759172 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:18.759172 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:18.759172 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:18.759172 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 318/616] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:18.764194 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e85680-ea66-4a0e-a61a-bd4c787bdac9 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:18.895448 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:18.904981 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e85680-ea66-4a0e-a61a-bd4c787bdac9 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:18.952527 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2110aef0-e789-41d1-b91c-556a75a0b400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:18.953680 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110aef0-e789-41d1-b91c-556a75a0b400 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:18.953958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110aef0-e789-41d1-b91c-556a75a0b400 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:18.954302 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110aef0-e789-41d1-b91c-556a75a0b400 None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:18.954862 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2110aef0-e789-41d1-b91c-556a75a0b400 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:18.955162 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2110aef0-e789-41d1-b91c-556a75a0b400 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:18.956550 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2110aef0-e789-41d1-b91c-556a75a0b400 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:18.971555 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 299/617] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:18.976853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14faadcf-bebc-4c47-b759-a9dc43708ce7 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:19.000247 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc00fd9-bcc2-4d40-ae97-4a3d120002d8 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:19.000605 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc00fd9-bcc2-4d40-ae97-4a3d120002d8 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:19.000831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc00fd9-bcc2-4d40-ae97-4a3d120002d8 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:19.003357 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 300/618] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:35:19.007472 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc02887-5c4d-4b23-a1cf-deb97b73ced7 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:19.007758 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc02887-5c4d-4b23-a1cf-deb97b73ced7 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:19.007926 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc02887-5c4d-4b23-a1cf-deb97b73ced7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:19.026410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-ebc02887-5c4d-4b23-a1cf-deb97b73ced7 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:19.027599 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-ebc02887-5c4d-4b23-a1cf-deb97b73ced7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:19.104605 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 301/619] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:35:19.134745 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14faadcf-bebc-4c47-b759-a9dc43708ce7 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:19.162283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:19.182602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56372b64-499f-4bbf-b04b-fb4ebf676c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:19.183700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56372b64-499f-4bbf-b04b-fb4ebf676c0a None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:19.184178 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56372b64-499f-4bbf-b04b-fb4ebf676c0a None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:19.184703 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56372b64-499f-4bbf-b04b-fb4ebf676c0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:19.195966 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56372b64-499f-4bbf-b04b-fb4ebf676c0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:19.197731 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56372b64-499f-4bbf-b04b-fb4ebf676c0a None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:19.213353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-56372b64-499f-4bbf-b04b-fb4ebf676c0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:19.213353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:19.213353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:19.213353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:19.213353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:19.213353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:19.213353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:19.213353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:19.213353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:19.213353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:19.213353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:19.213353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:19.213353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:35:19.213353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:35:19.213353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:35:19.214359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:35:19.214359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:19.214359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:19.214359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:19.214359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:19.214359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:19.214359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:19.214359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:19.214359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:19.214359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:19.214359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:19.214359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:19.214359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:19.214359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:19.214359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:19.214359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:19.214359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:19.215248 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:35:19.215248 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:35:19.215248 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:35:19.215248 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:35:19.215248 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:35:19.215248 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:35:19.215248 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:19.215248 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:19.215248 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 319/620] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:19.220263 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc02887-5c4d-4b23-a1cf-deb97b73ced7 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:19.359333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:19.380460 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc02887-5c4d-4b23-a1cf-deb97b73ced7 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:19.430892 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbd7e4-43ab-4991-97dc-c5d7162053ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:19.434564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbd7e4-43ab-4991-97dc-c5d7162053ba None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:19.434564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbd7e4-43ab-4991-97dc-c5d7162053ba None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:19.434564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbd7e4-43ab-4991-97dc-c5d7162053ba None admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:19.434564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bbd7e4-43ab-4991-97dc-c5d7162053ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:19.434886 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bbd7e4-43ab-4991-97dc-c5d7162053ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:19.440513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bbd7e4-43ab-4991-97dc-c5d7162053ba None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:19.460240 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56372b64-499f-4bbf-b04b-fb4ebf676c0a None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:19.466350 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 302/621] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:35:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:19.472066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbd7e4-43ab-4991-97dc-c5d7162053ba None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:19.591532 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56372b64-499f-4bbf-b04b-fb4ebf676c0a None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:19.630662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ccf337-d153-4c3e-96f4-bbea70ba8549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:19.631657 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccf337-d153-4c3e-96f4-bbea70ba8549 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:19.632147 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccf337-d153-4c3e-96f4-bbea70ba8549 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:19.632668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccf337-d153-4c3e-96f4-bbea70ba8549 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:19.633135 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbd7e4-43ab-4991-97dc-c5d7162053ba None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:19.644752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ccf337-d153-4c3e-96f4-bbea70ba8549 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:19.650450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-23ccf337-d153-4c3e-96f4-bbea70ba8549 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:19.650450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:19.650450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:19.650450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:19.650450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:19.650450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:19.650450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:19.650450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:19.650450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:19.650450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:19.650450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:19.650450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:19.650450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:19.650450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:19.650450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:19.650450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:19.651419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:19.651419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:19.651419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:19.651419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:19.651419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:19.651419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:19.651419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:19.651419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:19.651419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:19.651419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:19.651419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:19.651419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:19.651419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:19.651419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:19.651419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:19.651419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:19.652301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:19.652301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:19.652301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:19.652301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:19.652301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:19.652301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:19.652301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:19.652301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:19.652301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:19.652301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:19.652301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:19.652301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:19.652301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 320/622] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Aug 26 00:35:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:19.654006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30f757-9b8a-45de-b563-bbcc45702ea0 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:19.654359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30f757-9b8a-45de-b563-bbcc45702ea0 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:19.654706 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30f757-9b8a-45de-b563-bbcc45702ea0 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:19.656957 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 321/623] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:19.661918 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b30f757-9b8a-45de-b563-bbcc45702ea0 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:19.673587 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024304a-e191-418f-8e06-3c2d7af69034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:19.674663 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024304a-e191-418f-8e06-3c2d7af69034 None admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:19.675200 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024304a-e191-418f-8e06-3c2d7af69034 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:19.675657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024304a-e191-418f-8e06-3c2d7af69034 None admin] PATH_INFO: `/v3/projects/fde7ac3ed472433ab3bba56e5e091510/users/2f700502671446c797a566f07398a9fc/roles/bce78c7e0a534135a820de7f3588aa4d` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:19.688533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:19.690397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:19.693114 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024304a-e191-418f-8e06-3c2d7af69034 None admin] RBAC: Authorizing `identity:create_grant(project_id=fde7ac3ed472433ab3bba56e5e091510, user_id=2f700502671446c797a566f07398a9fc, role_id=bce78c7e0a534135a820de7f3588aa4d)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:19.694746 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024304a-e191-418f-8e06-3c2d7af69034 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:19.767947 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b30f757-9b8a-45de-b563-bbcc45702ea0 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:19.796770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 303/624] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Aug 26 00:35:19 2022] PUT /identity/v3/projects/fde7ac3ed472433ab3bba56e5e091510/users/2f700502671446c797a566f07398a9fc/roles/bce78c7e0a534135a820de7f3588aa4d => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:35:19.798583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa0758-9fb9-47a0-a07a-0bc5a3fe3620 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:19.798698 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa0758-9fb9-47a0-a07a-0bc5a3fe3620 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:19.799095 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa0758-9fb9-47a0-a07a-0bc5a3fe3620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:19.811582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558e478-fd3b-4b3c-b52f-92bab15e7dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:19.812729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558e478-fd3b-4b3c-b52f-92bab15e7dee None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:19.813230 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558e478-fd3b-4b3c-b52f-92bab15e7dee None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:19.813742 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558e478-fd3b-4b3c-b52f-92bab15e7dee None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:19.815050 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558e478-fd3b-4b3c-b52f-92bab15e7dee None admin] RBAC: Adding query filter params (name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:19.816637 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558e478-fd3b-4b3c-b52f-92bab15e7dee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:19.818381 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558e478-fd3b-4b3c-b52f-92bab15e7dee None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:19.830336 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 322/625] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Aug 26 00:35:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:19.836524 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558e478-fd3b-4b3c-b52f-92bab15e7dee None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:19.840312 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-42fa0758-9fb9-47a0-a07a-0bc5a3fe3620 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:19.841568 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-42fa0758-9fb9-47a0-a07a-0bc5a3fe3620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:19.974433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 304/626] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:35:19.980996 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:19.997184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558e478-fd3b-4b3c-b52f-92bab15e7dee None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:20.034479 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632ddc5-972c-4ac1-ae3c-7edcb2642ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:20.035117 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632ddc5-972c-4ac1-ae3c-7edcb2642ddc None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:20.035401 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632ddc5-972c-4ac1-ae3c-7edcb2642ddc None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:20.035677 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632ddc5-972c-4ac1-ae3c-7edcb2642ddc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:20.046221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632ddc5-972c-4ac1-ae3c-7edcb2642ddc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:20.048246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632ddc5-972c-4ac1-ae3c-7edcb2642ddc None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:20.063769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-5632ddc5-972c-4ac1-ae3c-7edcb2642ddc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:20.063769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:20.063769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:20.063769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:20.063769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:20.063769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:20.063769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:20.063769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:20.063769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:20.063769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:20.063769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:20.063769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:20.063769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:20.063769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:20.063769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:20.063769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:20.065283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:20.065283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:20.065283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:20.065283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:20.065283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:20.065283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:20.065283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:20.065283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:20.065283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:20.065283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:20.065283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:20.065283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:20.065283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:20.065283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:20.065283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:20.065283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:20.065283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:20.066457 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:20.066457 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:20.066457 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:20.066457 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 323/627] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:20.073004 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa0758-9fb9-47a0-a07a-0bc5a3fe3620 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:20.204153 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa0758-9fb9-47a0-a07a-0bc5a3fe3620 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:20.235818 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:20.247311 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8f644-baa0-49a5-a8a7-6efea234f4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:20.247927 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8f644-baa0-49a5-a8a7-6efea234f4fa None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:20.248203 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8f644-baa0-49a5-a8a7-6efea234f4fa None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:20.248488 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8f644-baa0-49a5-a8a7-6efea234f4fa None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:20.251436 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f8f644-baa0-49a5-a8a7-6efea234f4fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:20.251436 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f8f644-baa0-49a5-a8a7-6efea234f4fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:20.251436 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f8f644-baa0-49a5-a8a7-6efea234f4fa None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:20.267254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 305/628] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:20.268753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632ddc5-972c-4ac1-ae3c-7edcb2642ddc None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:20.276286 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8f644-baa0-49a5-a8a7-6efea234f4fa None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:20.392934 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8f644-baa0-49a5-a8a7-6efea234f4fa None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:20.420546 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632ddc5-972c-4ac1-ae3c-7edcb2642ddc None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:20.431288 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f9f69-163b-4818-a51e-72649ee964fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:20.432334 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f9f69-163b-4818-a51e-72649ee964fa None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:20.432665 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f9f69-163b-4818-a51e-72649ee964fa None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:20.432945 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f9f69-163b-4818-a51e-72649ee964fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:20.447171 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f9f69-163b-4818-a51e-72649ee964fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:20.455237 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-e28f9f69-163b-4818-a51e-72649ee964fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:20.455237 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:20.455237 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:20.455237 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:20.455237 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:20.455237 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:20.455237 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:20.455237 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:20.455237 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:20.455237 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:20.455237 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:20.455237 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:20.455237 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:20.455237 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:20.455237 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:20.455237 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:20.456310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:20.456310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:20.456310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:20.456310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:20.456310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:20.456310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:20.456310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:20.456310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:20.456310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:20.456310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:20.456310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:20.456310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:20.456310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:20.456310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:20.456310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:20.456310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:20.462010 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:20.462010 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:20.462010 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:20.462010 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:20.462010 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:20.462010 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:20.462010 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:20.462010 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:20.462010 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:20.462010 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:20.462010 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:20.462010 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:20.462010 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 306/629] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Aug 26 00:35:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:20.462010 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac0097-5ac5-4703-8010-bcc5c7bf4ce2 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:20.462770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac0097-5ac5-4703-8010-bcc5c7bf4ce2 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:20.462770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac0097-5ac5-4703-8010-bcc5c7bf4ce2 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:20.463694 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 307/630] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:20.464508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac0097-5ac5-4703-8010-bcc5c7bf4ce2 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:20.467092 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed432416-861e-4ea6-b5af-0cd7db92a8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:20.467978 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed432416-861e-4ea6-b5af-0cd7db92a8c9 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:20.469091 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed432416-861e-4ea6-b5af-0cd7db92a8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:20.469437 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed432416-861e-4ea6-b5af-0cd7db92a8c9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:20.477947 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed432416-861e-4ea6-b5af-0cd7db92a8c9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:20.479486 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed432416-861e-4ea6-b5af-0cd7db92a8c9 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:20.481561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:20.497195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-ed432416-861e-4ea6-b5af-0cd7db92a8c9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 00:35:20.497195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:20.497195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:20.497195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:20.497195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:20.497195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:20.497195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:20.497195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:20.497195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:20.497195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:20.497195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:20.497195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:20.497195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:35:20.497195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:35:20.497195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:35:20.497195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:35:20.498196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:20.498196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:20.498196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:20.498196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:20.498196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:20.498196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:20.498196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:20.498196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:20.498196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:20.498196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:20.498196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:20.498196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:20.498196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:20.498196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:20.498196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:20.498196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:20.498196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:20.498196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:20.499201 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:20.499201 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:20.499201 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:35:20.499201 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:35:20.499201 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:35:20.499201 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:35:20.499201 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:35:20.499201 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:35:20.499201 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 00:35:20.499201 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:20.499201 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 324/631] 10.222.0.59 () {56 vars in 1093 bytes} [Fri Aug 26 00:35:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:20.499201 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f1ba8-e559-41a1-ac34-92557ca0afd7 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:20.499201 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f1ba8-e559-41a1-ac34-92557ca0afd7 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:20.499953 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f1ba8-e559-41a1-ac34-92557ca0afd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:20.513256 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-d73f1ba8-e559-41a1-ac34-92557ca0afd7 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:20.514597 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-d73f1ba8-e559-41a1-ac34-92557ca0afd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:20.522260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:20.591665 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 325/632] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:35:20.592759 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f1ba8-e559-41a1-ac34-92557ca0afd7 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:20.629184 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac0097-5ac5-4703-8010-bcc5c7bf4ce2 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:20.695896 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f1ba8-e559-41a1-ac34-92557ca0afd7 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:20.728334 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6eabf-a040-47ac-b095-56ffd4efb03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:20.729081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6eabf-a040-47ac-b095-56ffd4efb03a None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:20.729363 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6eabf-a040-47ac-b095-56ffd4efb03a None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:20.729632 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6eabf-a040-47ac-b095-56ffd4efb03a None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:20.730299 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6eabf-a040-47ac-b095-56ffd4efb03a None admin] RBAC: Adding query filter params (name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:20.730600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6eabf-a040-47ac-b095-56ffd4efb03a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:20.731971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:20.732767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6eabf-a040-47ac-b095-56ffd4efb03a None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:20.740617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a838da3-7834-4e16-96bb-6125274285d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:20.742511 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a838da3-7834-4e16-96bb-6125274285d3 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:20.742511 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a838da3-7834-4e16-96bb-6125274285d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:20.742511 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a838da3-7834-4e16-96bb-6125274285d3 None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:20.742746 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 308/633] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Aug 26 00:35:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:20.743558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f57c6a-26cf-47a6-b096-74327526d7c2 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:20.743912 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f57c6a-26cf-47a6-b096-74327526d7c2 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:20.744257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f57c6a-26cf-47a6-b096-74327526d7c2 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:20.745554 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 309/634] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:20.750196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a838da3-7834-4e16-96bb-6125274285d3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:20.750196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a838da3-7834-4e16-96bb-6125274285d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:20.750196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a838da3-7834-4e16-96bb-6125274285d3 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:20.751331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f57c6a-26cf-47a6-b096-74327526d7c2 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:20.773405 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 326/635] 10.222.0.59 () {56 vars in 1133 bytes} [Fri Aug 26 00:35:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:20.773405 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a376f-5d8d-436e-a046-d42fa0d4baac None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:20.773405 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a376f-5d8d-436e-a046-d42fa0d4baac None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:20.773405 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a376f-5d8d-436e-a046-d42fa0d4baac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:20.790625 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-472a376f-5d8d-436e-a046-d42fa0d4baac None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:20.791905 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-472a376f-5d8d-436e-a046-d42fa0d4baac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:20.861937 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 327/636] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:35:20.863063 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a376f-5d8d-436e-a046-d42fa0d4baac None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:20.875865 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f57c6a-26cf-47a6-b096-74327526d7c2 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:20.878936 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:20.912994 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a422e4-51b9-4034-84f9-84ee84cca223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:20.913664 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a422e4-51b9-4034-84f9-84ee84cca223 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:20.913941 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a422e4-51b9-4034-84f9-84ee84cca223 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:20.914274 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a422e4-51b9-4034-84f9-84ee84cca223 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:20.924744 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a422e4-51b9-4034-84f9-84ee84cca223 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:20.926245 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a422e4-51b9-4034-84f9-84ee84cca223 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:20.942133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-53a422e4-51b9-4034-84f9-84ee84cca223 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:20.942133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:20.942133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:20.942133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:20.942133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:20.942133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:20.942133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:20.942133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:20.942133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:20.942133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:20.942133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:20.942133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:20.942133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:20.942133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:20.942133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:20.942133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:20.943127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:20.943127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:20.943127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:20.943127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:20.943127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:20.943127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:20.943127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:20.943127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:20.943127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:20.943127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:20.943127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:20.943127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:20.943127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:20.943127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:20.943127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:20.943127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:20.943127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:20.944411 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:20.944411 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:20.944411 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:20.944411 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 310/637] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:20.944411 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a422e4-51b9-4034-84f9-84ee84cca223 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:20.971129 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a376f-5d8d-436e-a046-d42fa0d4baac None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:21.018583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:21.021943 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196fb6d9-8bf9-4a9a-86af-1fa50aa5d10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:21.022567 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196fb6d9-8bf9-4a9a-86af-1fa50aa5d10d None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:21.022849 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196fb6d9-8bf9-4a9a-86af-1fa50aa5d10d None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:21.023124 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196fb6d9-8bf9-4a9a-86af-1fa50aa5d10d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:21.061809 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a422e4-51b9-4034-84f9-84ee84cca223 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:21.096105 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196fb6d9-8bf9-4a9a-86af-1fa50aa5d10d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:21.098264 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196fb6d9-8bf9-4a9a-86af-1fa50aa5d10d None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:21.106747 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd77a6-ae5f-409d-b80b-0fab5df4f8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:21.107827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd77a6-ae5f-409d-b80b-0fab5df4f8a0 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:21.108114 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd77a6-ae5f-409d-b80b-0fab5df4f8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:21.108393 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd77a6-ae5f-409d-b80b-0fab5df4f8a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:21.122303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd77a6-ae5f-409d-b80b-0fab5df4f8a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:21.129631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-ffbd77a6-ae5f-409d-b80b-0fab5df4f8a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:21.129631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:21.129631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:21.129631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:21.129631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:21.129631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:21.129631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:21.129631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:21.129631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:21.129631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:21.129631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:21.129631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:21.129631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:21.129631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:21.129631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:21.129631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:21.130597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:21.130597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:21.130597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:21.130597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:21.130597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:21.130597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:21.130597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:21.130597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:21.130597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:21.130597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:21.130597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:21.130597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:21.130597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:21.130597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:21.130597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:21.130597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:21.131438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:21.131438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:21.131438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:21.131438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:21.131438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:21.131438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:21.131438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:21.131438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:21.131438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:21.131438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:21.131438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:21.131438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:21.131438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 311/638] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Aug 26 00:35:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:21.132172 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd77a6-ae5f-409d-b80b-0fab5df4f8a0 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:21.156428 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:21.216861 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 328/639] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:35:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:35:21.218335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196fb6d9-8bf9-4a9a-86af-1fa50aa5d10d None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:21.237668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:21.250958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd77a6-ae5f-409d-b80b-0fab5df4f8a0 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:21.283506 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51846f59-2ff2-4633-9aa2-ca581e980e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:21.284043 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51846f59-2ff2-4633-9aa2-ca581e980e22 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:21.284325 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51846f59-2ff2-4633-9aa2-ca581e980e22 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:21.284634 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51846f59-2ff2-4633-9aa2-ca581e980e22 None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:21.285276 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51846f59-2ff2-4633-9aa2-ca581e980e22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:21.285591 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51846f59-2ff2-4633-9aa2-ca581e980e22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:21.286951 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51846f59-2ff2-4633-9aa2-ca581e980e22 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:21.302694 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 312/640] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:21.302694 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51846f59-2ff2-4633-9aa2-ca581e980e22 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:21.324009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196fb6d9-8bf9-4a9a-86af-1fa50aa5d10d None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:21.357207 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb35db5-561c-4666-a5c5-9b72f29b582f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:21.357917 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb35db5-561c-4666-a5c5-9b72f29b582f None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:21.358203 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb35db5-561c-4666-a5c5-9b72f29b582f None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:21.358478 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb35db5-561c-4666-a5c5-9b72f29b582f None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:21.359699 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb35db5-561c-4666-a5c5-9b72f29b582f None admin] RBAC: Adding query filter params (name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:21.360007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb35db5-561c-4666-a5c5-9b72f29b582f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:21.361406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb35db5-561c-4666-a5c5-9b72f29b582f None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:21.370927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 329/641] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Aug 26 00:35:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:21.371869 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb35db5-561c-4666-a5c5-9b72f29b582f None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:21.399979 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51846f59-2ff2-4633-9aa2-ca581e980e22 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:21.435420 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2d569-0ad6-4be9-9025-143312b4b80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:21.436058 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2d569-0ad6-4be9-9025-143312b4b80c None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:21.436340 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2d569-0ad6-4be9-9025-143312b4b80c None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:21.436648 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2d569-0ad6-4be9-9025-143312b4b80c None admin] PATH_INFO: `/v3/roles/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:21.447959 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d2d569-0ad6-4be9-9025-143312b4b80c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:21.454066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-e6d2d569-0ad6-4be9-9025-143312b4b80c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:21.454066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:21.454066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:21.454066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:21.454066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:21.454066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:21.454066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:21.454066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:21.454066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:21.454066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:21.454066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:21.454066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:21.454066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:21.454066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:21.454066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:21.454066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:21.455071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:21.455071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:21.455071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:21.455071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:21.455071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:21.455071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:21.455071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:21.455071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:21.455071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:21.455071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:21.455071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:21.455071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:21.455071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:21.455071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:21.455071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:21.455071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:21.455984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:21.455984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:21.455984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:21.455984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:21.455984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:21.455984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:21.455984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:21.455984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:21.455984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:21.455984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:21.455984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:21.455984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:21.455984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 313/642] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Aug 26 00:35:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:21.455984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2d569-0ad6-4be9-9025-143312b4b80c None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:21.496545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb35db5-561c-4666-a5c5-9b72f29b582f None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:21.509290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:21.532852 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6625111-ae6d-4c7a-8e44-8e44b3f25b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:21.533881 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6625111-ae6d-4c7a-8e44-8e44b3f25b85 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:21.534174 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6625111-ae6d-4c7a-8e44-8e44b3f25b85 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:21.534486 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6625111-ae6d-4c7a-8e44-8e44b3f25b85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:21.543000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:21.547124 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6625111-ae6d-4c7a-8e44-8e44b3f25b85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:21.549120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6625111-ae6d-4c7a-8e44-8e44b3f25b85 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:21.557584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2d569-0ad6-4be9-9025-143312b4b80c None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:21.562710 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-a6625111-ae6d-4c7a-8e44-8e44b3f25b85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:21.562710 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:21.562710 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:21.562710 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:21.562710 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:21.562710 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:21.562710 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:21.562710 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:21.562710 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:21.562710 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:21.562710 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:21.562710 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:21.562710 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:21.562710 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:21.562710 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:21.562710 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:21.563773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:21.563773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:21.563773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:21.563773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:21.563773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:21.563773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:21.563773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:21.563773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:21.563773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:21.563773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:21.563773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:21.563773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:21.563773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:21.563773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:21.563773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:21.563773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:21.563773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:21.564721 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:21.564721 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:21.564721 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:21.568540 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 330/643] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:21.569200 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6625111-ae6d-4c7a-8e44-8e44b3f25b85 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:21.595668 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e864da5-734f-48f1-b6af-c49b3eeeb32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:21.596305 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e864da5-734f-48f1-b6af-c49b3eeeb32e None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:21.596619 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e864da5-734f-48f1-b6af-c49b3eeeb32e None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:21.596899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e864da5-734f-48f1-b6af-c49b3eeeb32e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:21.599643 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:21.604979 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e864da5-734f-48f1-b6af-c49b3eeeb32e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:21.606680 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e864da5-734f-48f1-b6af-c49b3eeeb32e None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:21.618480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-2e864da5-734f-48f1-b6af-c49b3eeeb32e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 00:35:21.618480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:21.618480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:21.618480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:21.618480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:21.618480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:21.618480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:21.618480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:21.618480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:21.618480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:21.618480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:21.618480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:21.618480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:35:21.618480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:35:21.618480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:35:21.618480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:35:21.619626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:21.619626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:21.619626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:21.619626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:21.619626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:21.619626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:21.619626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:21.619626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:21.619626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:21.619626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:21.619626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:21.619626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:21.619626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:21.619626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:21.619626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:21.619626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:21.619626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:21.619626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:21.621222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:21.621222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:21.621222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:35:21.621222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:35:21.621222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:35:21.621222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:35:21.621222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:35:21.621222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:35:21.621222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 00:35:21.621222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:21.621222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 314/644] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:35:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:21.621222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e864da5-734f-48f1-b6af-c49b3eeeb32e None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:21.680411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6625111-ae6d-4c7a-8e44-8e44b3f25b85 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:21.713305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:21.721878 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c75dd-e34a-4ffe-8ab0-a2a93e0dfc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:21.722783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c75dd-e34a-4ffe-8ab0-a2a93e0dfc28 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:21.723061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c75dd-e34a-4ffe-8ab0-a2a93e0dfc28 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:21.723334 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c75dd-e34a-4ffe-8ab0-a2a93e0dfc28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:21.724261 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e864da5-734f-48f1-b6af-c49b3eeeb32e None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:21.737282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c75dd-e34a-4ffe-8ab0-a2a93e0dfc28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:21.739483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c75dd-e34a-4ffe-8ab0-a2a93e0dfc28 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:21.754408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-cc0c75dd-e34a-4ffe-8ab0-a2a93e0dfc28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:21.754408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:21.754408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:21.754408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:21.754408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:21.754408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:21.754408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:21.754408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:21.754408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:21.754408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:21.754408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:21.754408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:21.754408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:21.754408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:21.754408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:21.754408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:21.755402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:21.755402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:21.755402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:21.755402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:21.755402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:21.755402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:21.755402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:21.755402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:21.755402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:21.755402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:21.755402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:21.755402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:21.755402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:21.755402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:21.755402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:21.755402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:21.755402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:21.756325 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:21.756325 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:21.756325 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:21.756325 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 331/645] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:21.756325 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c75dd-e34a-4ffe-8ab0-a2a93e0dfc28 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:21.764029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa934670-c2b1-45aa-9397-b1279a800374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:21.764029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa934670-c2b1-45aa-9397-b1279a800374 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:21.764303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa934670-c2b1-45aa-9397-b1279a800374 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:21.764395 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa934670-c2b1-45aa-9397-b1279a800374 None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:21.765602 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa934670-c2b1-45aa-9397-b1279a800374 None admin] RBAC: Adding query filter params (name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:21.765917 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa934670-c2b1-45aa-9397-b1279a800374 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:21.767299 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa934670-c2b1-45aa-9397-b1279a800374 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:21.775724 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 315/646] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Aug 26 00:35:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:21.775724 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa934670-c2b1-45aa-9397-b1279a800374 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:21.837651 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:21.877297 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c75dd-e34a-4ffe-8ab0-a2a93e0dfc28 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:21.883791 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa934670-c2b1-45aa-9397-b1279a800374 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:21.913509 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77921006-e327-4382-8bab-4b6c828ae9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:21.914125 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77921006-e327-4382-8bab-4b6c828ae9a7 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:21.914409 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77921006-e327-4382-8bab-4b6c828ae9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:21.914690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77921006-e327-4382-8bab-4b6c828ae9a7 None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:21.915331 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77921006-e327-4382-8bab-4b6c828ae9a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:21.915635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77921006-e327-4382-8bab-4b6c828ae9a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:21.917039 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77921006-e327-4382-8bab-4b6c828ae9a7 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:21.921865 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8b1c5-4f26-4ae8-8885-e9e61f3b7390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:21.922465 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8b1c5-4f26-4ae8-8885-e9e61f3b7390 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:21.922742 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8b1c5-4f26-4ae8-8885-e9e61f3b7390 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:21.923009 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8b1c5-4f26-4ae8-8885-e9e61f3b7390 None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:21.923746 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce8b1c5-4f26-4ae8-8885-e9e61f3b7390 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:21.924048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce8b1c5-4f26-4ae8-8885-e9e61f3b7390 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:21.926402 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce8b1c5-4f26-4ae8-8885-e9e61f3b7390 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:21.929509 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 332/647] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:21.929845 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77921006-e327-4382-8bab-4b6c828ae9a7 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:21.952522 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 316/648] 10.222.0.59 () {56 vars in 1137 bytes} [Fri Aug 26 00:35:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:21.952522 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8b1c5-4f26-4ae8-8885-e9e61f3b7390 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:21.974778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:22.029728 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77921006-e327-4382-8bab-4b6c828ae9a7 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:22.049734 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8b1c5-4f26-4ae8-8885-e9e61f3b7390 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:22.063803 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c411e8-1638-4ab1-a7d3-b0927cb24e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:22.069373 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c411e8-1638-4ab1-a7d3-b0927cb24e76 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:22.069373 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c411e8-1638-4ab1-a7d3-b0927cb24e76 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:22.069373 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c411e8-1638-4ab1-a7d3-b0927cb24e76 None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:22.069373 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c411e8-1638-4ab1-a7d3-b0927cb24e76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:22.069734 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:22.069841 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c411e8-1638-4ab1-a7d3-b0927cb24e76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:22.069841 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c411e8-1638-4ab1-a7d3-b0927cb24e76 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:22.079314 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 333/649] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:22.080136 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c411e8-1638-4ab1-a7d3-b0927cb24e76 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:22.084166 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c7619-0c24-410b-998e-9498d672fa97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:22.084795 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c7619-0c24-410b-998e-9498d672fa97 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:22.085074 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c7619-0c24-410b-998e-9498d672fa97 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:22.085340 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c7619-0c24-410b-998e-9498d672fa97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:22.095590 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c7619-0c24-410b-998e-9498d672fa97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:22.097002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c7619-0c24-410b-998e-9498d672fa97 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:22.110849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-487c7619-0c24-410b-998e-9498d672fa97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:22.110849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:22.110849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:22.110849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:22.110849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:22.110849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:22.110849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:22.110849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:22.110849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:22.110849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:22.110849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:22.110849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:22.110849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:22.110849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:22.110849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:22.110849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:22.111999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:22.111999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:22.111999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:22.111999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:22.111999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:22.111999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:22.111999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:22.111999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:22.111999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:22.111999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:22.111999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:22.111999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:22.111999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:22.111999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:22.111999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:22.111999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:22.111999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:22.113726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:22.113726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:22.113726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:22.116928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 317/650] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:22.118067 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3007293-42db-4e10-821c-bed3b3a6bc41 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:22.118502 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3007293-42db-4e10-821c-bed3b3a6bc41 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:22.118780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3007293-42db-4e10-821c-bed3b3a6bc41 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:22.120034 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 318/651] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:22.120759 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3007293-42db-4e10-821c-bed3b3a6bc41 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:22.183786 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c411e8-1638-4ab1-a7d3-b0927cb24e76 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:22.185295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:22.218593 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb1ff5d-d67a-4e38-a6a3-166c4e6fceb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:22.219060 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3007293-42db-4e10-821c-bed3b3a6bc41 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:22.219234 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1ff5d-d67a-4e38-a6a3-166c4e6fceb0 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:22.219517 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1ff5d-d67a-4e38-a6a3-166c4e6fceb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:22.219791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1ff5d-d67a-4e38-a6a3-166c4e6fceb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:22.226547 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb1ff5d-d67a-4e38-a6a3-166c4e6fceb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:22.228148 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb1ff5d-d67a-4e38-a6a3-166c4e6fceb0 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:22.239169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-bbb1ff5d-d67a-4e38-a6a3-166c4e6fceb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:22.239169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:22.239169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:22.239169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:22.239169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:22.239169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:22.239169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:22.239169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:22.239169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:22.239169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:22.239169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:22.239169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:22.239169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:35:22.239169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:35:22.239169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:35:22.240226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:35:22.240226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:22.240226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:22.240226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:22.240226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:22.240226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:22.240226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:22.240226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:22.240226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:22.240226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:22.240226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:22.240226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:22.240226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:22.240226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:22.240226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:22.240226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:22.240226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:22.241197 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:35:22.241197 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:35:22.241197 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:35:22.241197 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:35:22.241197 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:35:22.241197 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:35:22.241197 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:22.241197 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:22.241197 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 334/652] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:22.241712 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb1ff5d-d67a-4e38-a6a3-166c4e6fceb0 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:22.256195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37df9f9-3ccc-4ef7-8c53-85c9919b8318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:22.256821 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37df9f9-3ccc-4ef7-8c53-85c9919b8318 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:22.257097 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37df9f9-3ccc-4ef7-8c53-85c9919b8318 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:22.257364 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37df9f9-3ccc-4ef7-8c53-85c9919b8318 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:22.291117 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:22.323235 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37df9f9-3ccc-4ef7-8c53-85c9919b8318 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:22.324706 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37df9f9-3ccc-4ef7-8c53-85c9919b8318 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:22.340645 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb1ff5d-d67a-4e38-a6a3-166c4e6fceb0 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:22.373486 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2508e8de-01d5-4659-9c1b-5bc8d06fe4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:22.374047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508e8de-01d5-4659-9c1b-5bc8d06fe4db None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:22.374333 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508e8de-01d5-4659-9c1b-5bc8d06fe4db None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:22.375110 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508e8de-01d5-4659-9c1b-5bc8d06fe4db None admin] PATH_INFO: `/v3/users/glance` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:22.381977 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2508e8de-01d5-4659-9c1b-5bc8d06fe4db None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:22.383400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2508e8de-01d5-4659-9c1b-5bc8d06fe4db None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:22.395155 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-2508e8de-01d5-4659-9c1b-5bc8d06fe4db None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 26 00:35:22.395155 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:22.395155 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:22.395155 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:22.395155 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:22.395155 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:22.395155 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:22.395155 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:22.395155 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:22.395155 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:22.395155 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:22.395155 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:22.395155 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:35:22.395155 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:35:22.395155 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:35:22.395155 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:35:22.396355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:22.396355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:22.396355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:22.396355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:22.396355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:22.396355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:22.396355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:22.396355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:22.396355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:22.396355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:22.396355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:22.396355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:22.396355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:22.396355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:22.396355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:22.396355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:22.396355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:22.396355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:22.397345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:22.397345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:22.397345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:35:22.397345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:35:22.397345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:35:22.397345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:35:22.397345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:35:22.397345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:35:22.397345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 26 00:35:22.397345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:22.397345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 335/653] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:35:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:22.397345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2508e8de-01d5-4659-9c1b-5bc8d06fe4db None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:22.420958 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:22.426650 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 319/654] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:35:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:35:22.427888 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bba591-eb21-4b4c-b27b-b0b6e5e2fe73 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:22.428186 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bba591-eb21-4b4c-b27b-b0b6e5e2fe73 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:22.428454 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bba591-eb21-4b4c-b27b-b0b6e5e2fe73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:22.446386 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-52bba591-eb21-4b4c-b27b-b0b6e5e2fe73 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:22.447580 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-52bba591-eb21-4b4c-b27b-b0b6e5e2fe73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:22.496713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2508e8de-01d5-4659-9c1b-5bc8d06fe4db None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:22.508807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:22.513733 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 320/655] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:35:22.514575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bba591-eb21-4b4c-b27b-b0b6e5e2fe73 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:22.542543 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591b113-3c42-4e82-90be-ccb2257399c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:22.543250 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591b113-3c42-4e82-90be-ccb2257399c5 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:22.543534 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591b113-3c42-4e82-90be-ccb2257399c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:22.543813 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591b113-3c42-4e82-90be-ccb2257399c5 None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:22.544544 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591b113-3c42-4e82-90be-ccb2257399c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:22.544852 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591b113-3c42-4e82-90be-ccb2257399c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:22.546233 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591b113-3c42-4e82-90be-ccb2257399c5 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:22.559514 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 336/656] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:22.565080 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591b113-3c42-4e82-90be-ccb2257399c5 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:22.617681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bba591-eb21-4b4c-b27b-b0b6e5e2fe73 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:22.648242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:22.654292 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55e31c-85be-44e8-9e41-b20d22da39a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:22.654907 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55e31c-85be-44e8-9e41-b20d22da39a4 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:22.655185 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55e31c-85be-44e8-9e41-b20d22da39a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:22.655523 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55e31c-85be-44e8-9e41-b20d22da39a4 None admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:22.656248 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55e31c-85be-44e8-9e41-b20d22da39a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:22.656590 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55e31c-85be-44e8-9e41-b20d22da39a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:22.658029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55e31c-85be-44e8-9e41-b20d22da39a4 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:22.666967 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591b113-3c42-4e82-90be-ccb2257399c5 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:22.673185 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 321/657] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:35:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:22.674007 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55e31c-85be-44e8-9e41-b20d22da39a4 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:22.705801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556992b-156d-4323-bfb2-10421e62f16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:22.706422 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556992b-156d-4323-bfb2-10421e62f16e None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:22.706708 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556992b-156d-4323-bfb2-10421e62f16e None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:22.706988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556992b-156d-4323-bfb2-10421e62f16e None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:22.707717 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556992b-156d-4323-bfb2-10421e62f16e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:22.708028 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556992b-156d-4323-bfb2-10421e62f16e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:22.709523 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556992b-156d-4323-bfb2-10421e62f16e None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:22.728829 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 337/658] 10.222.0.59 () {56 vars in 1137 bytes} [Fri Aug 26 00:35:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:22.729646 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556992b-156d-4323-bfb2-10421e62f16e None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:22.736739 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:22.778211 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55e31c-85be-44e8-9e41-b20d22da39a4 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:22.811783 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ee43f-e882-43ce-bad5-9d5141742c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:22.812435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ee43f-e882-43ce-bad5-9d5141742c57 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:22.812745 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ee43f-e882-43ce-bad5-9d5141742c57 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:22.813022 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ee43f-e882-43ce-bad5-9d5141742c57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:22.824490 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ee43f-e882-43ce-bad5-9d5141742c57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:22.825951 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ee43f-e882-43ce-bad5-9d5141742c57 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:22.836423 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556992b-156d-4323-bfb2-10421e62f16e None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:22.840169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-ac2ee43f-e882-43ce-bad5-9d5141742c57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:22.840169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:22.840169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:22.840169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:22.840169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:22.840169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:22.840169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:22.840169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:22.840169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:22.840169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:22.840169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:22.840169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:22.840169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:22.840169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:22.840169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:22.840169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:22.841336 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:22.841336 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:22.841336 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:22.841336 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:22.841336 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:22.841336 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:22.841336 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:22.841336 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:22.841336 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:22.841336 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:22.841336 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:22.841336 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:22.841336 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:22.841336 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:22.841336 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:22.841336 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:22.841336 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:22.842257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:22.842257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:22.842257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:22.842257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 322/659] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:22.842257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ee43f-e882-43ce-bad5-9d5141742c57 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:22.872761 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7118b398-0747-4204-a038-ebe8ef3274b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:22.873397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118b398-0747-4204-a038-ebe8ef3274b9 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:22.873677 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118b398-0747-4204-a038-ebe8ef3274b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:22.873950 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118b398-0747-4204-a038-ebe8ef3274b9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:22.881411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7118b398-0747-4204-a038-ebe8ef3274b9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:22.882891 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7118b398-0747-4204-a038-ebe8ef3274b9 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:22.885542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:22.896116 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-7118b398-0747-4204-a038-ebe8ef3274b9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 00:35:22.896116 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:22.896116 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:22.896116 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:22.896116 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:22.896116 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:22.896116 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:22.896116 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:22.896116 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:22.896116 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:22.896116 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:22.896116 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:22.896116 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:35:22.896116 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:35:22.896116 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:35:22.896116 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:35:22.897310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:22.897310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:22.897310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:22.897310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:22.897310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:22.897310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:22.897310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:22.897310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:22.897310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:22.897310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:22.897310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:22.897310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:22.897310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:22.897310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:22.897310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:22.897310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:22.897310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:22.897310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:22.914357 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:22.914357 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:22.914357 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:35:22.914357 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:35:22.914357 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:35:22.914357 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:35:22.914357 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:35:22.914357 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:35:22.914357 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 00:35:22.914357 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:22.914357 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 338/660] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Aug 26 00:35:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:22.914357 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7118b398-0747-4204-a038-ebe8ef3274b9 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:22.942549 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ee43f-e882-43ce-bad5-9d5141742c57 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:22.953626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:22.977028 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54614b97-6724-4139-816a-98bd427fcd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:22.977650 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54614b97-6724-4139-816a-98bd427fcd0b None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:22.977912 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54614b97-6724-4139-816a-98bd427fcd0b None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:22.978186 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54614b97-6724-4139-816a-98bd427fcd0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:22.978910 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54614b97-6724-4139-816a-98bd427fcd0b None admin] RBAC: Adding query filter params (role.id=bce78c7e0a534135a820de7f3588aa4d, scope.project.id=fde7ac3ed472433ab3bba56e5e091510, user.id=0602593263ca4b05a657d5b7697f932f) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:22.979216 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54614b97-6724-4139-816a-98bd427fcd0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:22.980673 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54614b97-6724-4139-816a-98bd427fcd0b None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:22.987324 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 323/661] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:35:22 2022] GET /identity/v3/role_assignments?user.id=0602593263ca4b05a657d5b7697f932f&scope.project.id=fde7ac3ed472433ab3bba56e5e091510&role.id=bce78c7e0a534135a820de7f3588aa4d => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:22.996786 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54614b97-6724-4139-816a-98bd427fcd0b None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:22.997598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7118b398-0747-4204-a038-ebe8ef3274b9 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:23.036328 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740d602-7b39-4e13-b557-6ec321d7c855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:23.037297 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740d602-7b39-4e13-b557-6ec321d7c855 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:23.037579 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740d602-7b39-4e13-b557-6ec321d7c855 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:23.037856 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740d602-7b39-4e13-b557-6ec321d7c855 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:23.108359 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54614b97-6724-4139-816a-98bd427fcd0b None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:23.112489 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:23.115144 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740d602-7b39-4e13-b557-6ec321d7c855 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:23.116715 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740d602-7b39-4e13-b557-6ec321d7c855 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:23.150349 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ea1b1-7bcb-46d9-8eb7-5d509e56792e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:23.150987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ea1b1-7bcb-46d9-8eb7-5d509e56792e None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:23.151284 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ea1b1-7bcb-46d9-8eb7-5d509e56792e None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:23.152217 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ea1b1-7bcb-46d9-8eb7-5d509e56792e None admin] PATH_INFO: `/v3/roles/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:23.163889 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ea1b1-7bcb-46d9-8eb7-5d509e56792e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:23.169613 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-5e3ea1b1-7bcb-46d9-8eb7-5d509e56792e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:23.169613 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:23.169613 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:23.169613 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:23.169613 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:23.169613 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:23.169613 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:23.169613 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:23.169613 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:23.169613 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:23.169613 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:23.169613 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:23.169613 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:23.169613 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:23.169613 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:23.169613 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:23.170600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:23.170600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:23.170600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:23.170600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:23.170600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:23.170600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:23.170600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:23.170600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:23.170600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:23.170600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:23.170600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:23.170600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:23.170600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:23.170600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:23.170600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:23.170600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:23.171884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:23.171884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:23.171884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:23.171884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:23.171884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:23.171884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:23.171884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:23.171884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:23.171884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:23.171884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:23.171884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:23.171884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:23.171884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 324/662] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Aug 26 00:35:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:23.171884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ea1b1-7bcb-46d9-8eb7-5d509e56792e None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:23.215350 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:23.244503 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 339/663] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:35:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:35:23.244503 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740d602-7b39-4e13-b557-6ec321d7c855 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:23.294033 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ea1b1-7bcb-46d9-8eb7-5d509e56792e None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:23.330031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3351043a-0b22-4d6c-b2d9-591002da2494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:23.330738 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351043a-0b22-4d6c-b2d9-591002da2494 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:23.331116 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351043a-0b22-4d6c-b2d9-591002da2494 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:23.331461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351043a-0b22-4d6c-b2d9-591002da2494 None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:23.332179 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3351043a-0b22-4d6c-b2d9-591002da2494 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:23.332589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3351043a-0b22-4d6c-b2d9-591002da2494 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:23.333962 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3351043a-0b22-4d6c-b2d9-591002da2494 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:23.349760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 325/664] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:23.349760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3351043a-0b22-4d6c-b2d9-591002da2494 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:23.355840 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:23.358760 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740d602-7b39-4e13-b557-6ec321d7c855 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:23.397903 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b200eb-9534-40ba-a254-ab685c06caac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:23.398533 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b200eb-9534-40ba-a254-ab685c06caac None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:23.398815 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b200eb-9534-40ba-a254-ab685c06caac None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:23.399089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b200eb-9534-40ba-a254-ab685c06caac None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:23.399820 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b200eb-9534-40ba-a254-ab685c06caac None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:23.400130 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b200eb-9534-40ba-a254-ab685c06caac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:23.403951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b200eb-9534-40ba-a254-ab685c06caac None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:23.426035 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 340/665] 10.222.0.59 () {56 vars in 1139 bytes} [Fri Aug 26 00:35:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:23.426850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b200eb-9534-40ba-a254-ab685c06caac None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:23.453259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:23.460609 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3351043a-0b22-4d6c-b2d9-591002da2494 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:23.496187 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068a009b-8d42-4faf-9a3c-58be44a38bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:23.496839 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a009b-8d42-4faf-9a3c-58be44a38bb5 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:23.497122 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a009b-8d42-4faf-9a3c-58be44a38bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:23.497394 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a009b-8d42-4faf-9a3c-58be44a38bb5 None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:23.498055 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068a009b-8d42-4faf-9a3c-58be44a38bb5 None admin] RBAC: Adding query filter params (name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:23.498358 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068a009b-8d42-4faf-9a3c-58be44a38bb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:23.500363 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068a009b-8d42-4faf-9a3c-58be44a38bb5 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:23.509143 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 326/666] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Aug 26 00:35:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:23.513882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068a009b-8d42-4faf-9a3c-58be44a38bb5 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:23.537914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b200eb-9534-40ba-a254-ab685c06caac None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:23.573290 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4daedb-0784-4648-99f4-41ec4da23864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:23.573926 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4daedb-0784-4648-99f4-41ec4da23864 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:23.574216 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4daedb-0784-4648-99f4-41ec4da23864 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:23.574483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4daedb-0784-4648-99f4-41ec4da23864 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:23.585312 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4daedb-0784-4648-99f4-41ec4da23864 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:23.586747 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4daedb-0784-4648-99f4-41ec4da23864 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:23.602052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-bd4daedb-0784-4648-99f4-41ec4da23864 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:23.602052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:23.602052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:23.602052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:23.602052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:23.602052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:23.602052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:23.602052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:23.602052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:23.602052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:23.602052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:23.602052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:23.602052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:23.602052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:23.602052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:23.602052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:23.603068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:23.603068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:23.603068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:23.603068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:23.603068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:23.603068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:23.603068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:23.603068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:23.603068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:23.603068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:23.603068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:23.603068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:23.603068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:23.603068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:23.603068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:23.603068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:23.603068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:23.603983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:23.603983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:23.603983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:23.603983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 341/667] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:23.603983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4daedb-0784-4648-99f4-41ec4da23864 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:23.617992 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:23.627940 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068a009b-8d42-4faf-9a3c-58be44a38bb5 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:23.666638 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c14e5b-f144-44c8-8751-4c644c285e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:23.667323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c14e5b-f144-44c8-8751-4c644c285e2c None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:23.667617 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c14e5b-f144-44c8-8751-4c644c285e2c None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:23.667876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c14e5b-f144-44c8-8751-4c644c285e2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:23.675355 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c14e5b-f144-44c8-8751-4c644c285e2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:23.676875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c14e5b-f144-44c8-8751-4c644c285e2c None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:23.687958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-33c14e5b-f144-44c8-8751-4c644c285e2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:23.687958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:23.687958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:23.687958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:23.687958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:23.687958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:23.687958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:23.687958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:23.687958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:23.687958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:23.687958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:23.687958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:23.687958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:35:23.687958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:35:23.687958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:35:23.688915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:35:23.688915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:23.688915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:23.688915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:23.688915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:23.688915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:23.688915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:23.688915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:23.688915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:23.688915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:23.688915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:23.688915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:23.688915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:23.688915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:23.688915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:23.688915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:23.688915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:23.689831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:35:23.689831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:35:23.689831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:35:23.689831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:35:23.689831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:35:23.689831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:35:23.689831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:23.689831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:23.689831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 327/668] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:23.690327 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c14e5b-f144-44c8-8751-4c644c285e2c None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:23.711313 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4daedb-0784-4648-99f4-41ec4da23864 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:23.712186 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:23.748029 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee568e2e-c5d7-4e48-adee-f3592d836f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:23.748661 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee568e2e-c5d7-4e48-adee-f3592d836f54 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:23.748943 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee568e2e-c5d7-4e48-adee-f3592d836f54 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:23.749213 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee568e2e-c5d7-4e48-adee-f3592d836f54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:23.792601 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c14e5b-f144-44c8-8751-4c644c285e2c None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:23.818116 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee568e2e-c5d7-4e48-adee-f3592d836f54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:23.819556 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee568e2e-c5d7-4e48-adee-f3592d836f54 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:23.827245 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85dc54c-e03e-4aa5-87bc-b28fa1c331f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:23.827919 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85dc54c-e03e-4aa5-87bc-b28fa1c331f8 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:23.828201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85dc54c-e03e-4aa5-87bc-b28fa1c331f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:23.828497 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85dc54c-e03e-4aa5-87bc-b28fa1c331f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:23.831843 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:23.838286 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85dc54c-e03e-4aa5-87bc-b28fa1c331f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:23.839713 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85dc54c-e03e-4aa5-87bc-b28fa1c331f8 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:23.854575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-a85dc54c-e03e-4aa5-87bc-b28fa1c331f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:23.854575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:23.854575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:23.854575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:23.854575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:23.854575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:23.854575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:23.854575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:23.854575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:23.854575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:23.854575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:23.854575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:23.854575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:23.854575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:23.854575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:23.854575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:23.855511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:23.855511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:23.855511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:23.855511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:23.855511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:23.855511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:23.855511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:23.855511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:23.855511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:23.855511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:23.855511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:23.855511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:23.855511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:23.855511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:23.855511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:23.855511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:23.855511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:23.856400 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:23.856400 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:23.856400 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:23.856400 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 328/669] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:23.856400 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85dc54c-e03e-4aa5-87bc-b28fa1c331f8 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:23.924381 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 342/670] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:35:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:35:23.925571 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee568e2e-c5d7-4e48-adee-f3592d836f54 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:23.930175 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:23.963438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85dc54c-e03e-4aa5-87bc-b28fa1c331f8 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:23.997160 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81c317-cb36-49ab-8522-c5a7ee01dac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:23.997763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81c317-cb36-49ab-8522-c5a7ee01dac3 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:23.998037 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81c317-cb36-49ab-8522-c5a7ee01dac3 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:23.998313 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81c317-cb36-49ab-8522-c5a7ee01dac3 None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:23.998947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81c317-cb36-49ab-8522-c5a7ee01dac3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:23.999243 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81c317-cb36-49ab-8522-c5a7ee01dac3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:24.001220 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81c317-cb36-49ab-8522-c5a7ee01dac3 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:24.013322 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 329/671] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:24.014091 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81c317-cb36-49ab-8522-c5a7ee01dac3 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:24.036541 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee568e2e-c5d7-4e48-adee-f3592d836f54 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:24.047938 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:24.074216 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b235a-79b2-4385-bdf6-e8ba83655ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:24.074827 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b235a-79b2-4385-bdf6-e8ba83655ff2 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:24.075111 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b235a-79b2-4385-bdf6-e8ba83655ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:24.075383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b235a-79b2-4385-bdf6-e8ba83655ff2 None admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:24.076164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b235a-79b2-4385-bdf6-e8ba83655ff2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:24.076466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b235a-79b2-4385-bdf6-e8ba83655ff2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:24.077941 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b235a-79b2-4385-bdf6-e8ba83655ff2 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:24.094662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 343/672] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:35:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:24.094662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b235a-79b2-4385-bdf6-e8ba83655ff2 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:24.121417 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81c317-cb36-49ab-8522-c5a7ee01dac3 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:24.152639 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:24.160855 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb89d8-404e-4771-90c8-90ff5acaa568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:24.168719 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb89d8-404e-4771-90c8-90ff5acaa568 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:24.168719 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb89d8-404e-4771-90c8-90ff5acaa568 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:24.168719 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb89d8-404e-4771-90c8-90ff5acaa568 None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:24.168719 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eb89d8-404e-4771-90c8-90ff5acaa568 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:24.168719 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eb89d8-404e-4771-90c8-90ff5acaa568 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:24.168719 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eb89d8-404e-4771-90c8-90ff5acaa568 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:24.180438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 330/673] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:24.180438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb89d8-404e-4771-90c8-90ff5acaa568 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:24.203489 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b235a-79b2-4385-bdf6-e8ba83655ff2 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:24.238620 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cbb6d4-2110-4182-91a9-e8846f3e3abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:24.239248 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cbb6d4-2110-4182-91a9-e8846f3e3abe None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:24.239525 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cbb6d4-2110-4182-91a9-e8846f3e3abe None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:24.239793 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cbb6d4-2110-4182-91a9-e8846f3e3abe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:24.251309 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cbb6d4-2110-4182-91a9-e8846f3e3abe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:24.252773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cbb6d4-2110-4182-91a9-e8846f3e3abe None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:24.266485 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-55cbb6d4-2110-4182-91a9-e8846f3e3abe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:24.266485 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:24.266485 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:24.266485 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:24.266485 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:24.266485 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:24.266485 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:24.266485 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:24.266485 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:24.266485 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:24.266485 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:24.266485 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:24.266485 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:24.266485 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:24.266485 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:24.266485 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:24.267456 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:24.267456 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:24.267456 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:24.267456 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:24.267456 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:24.267456 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:24.267456 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:24.267456 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:24.267456 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:24.267456 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:24.267456 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:24.267456 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:24.267456 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:24.267456 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:24.267456 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:24.267456 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:24.267456 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:24.268390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:24.268390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:24.268390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:24.268390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 344/674] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:24.268672 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cbb6d4-2110-4182-91a9-e8846f3e3abe None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:24.292280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:24.293533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb89d8-404e-4771-90c8-90ff5acaa568 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:24.330704 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21bab7-44fc-4e03-a37b-5a4b4b74c815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:24.331333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21bab7-44fc-4e03-a37b-5a4b4b74c815 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:24.331659 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21bab7-44fc-4e03-a37b-5a4b4b74c815 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:24.331927 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21bab7-44fc-4e03-a37b-5a4b4b74c815 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:24.338615 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e21bab7-44fc-4e03-a37b-5a4b4b74c815 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:24.340040 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e21bab7-44fc-4e03-a37b-5a4b4b74c815 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:24.351037 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-0e21bab7-44fc-4e03-a37b-5a4b4b74c815 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:24.351037 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:24.351037 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:24.351037 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:24.351037 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:24.351037 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:24.351037 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:24.351037 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:24.351037 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:24.351037 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:24.351037 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:24.351037 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:24.351037 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:35:24.351037 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:35:24.351037 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:35:24.351958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:35:24.351958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:24.351958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:24.351958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:24.351958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:24.351958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:24.351958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:24.351958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:24.351958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:24.351958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:24.351958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:24.351958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:24.351958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:24.351958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:24.351958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:24.351958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:24.351958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:24.352923 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:35:24.352923 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:35:24.352923 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:35:24.352923 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:35:24.352923 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:35:24.352923 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:35:24.352923 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:24.352923 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:24.352923 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 331/675] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:24.353430 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21bab7-44fc-4e03-a37b-5a4b4b74c815 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:24.381044 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cbb6d4-2110-4182-91a9-e8846f3e3abe None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:24.412183 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:24.418137 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c04af-950f-44fd-9f1c-44c4844d7484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:24.418752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c04af-950f-44fd-9f1c-44c4844d7484 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:24.419032 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c04af-950f-44fd-9f1c-44c4844d7484 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:24.419309 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c04af-950f-44fd-9f1c-44c4844d7484 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:24.420034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c04af-950f-44fd-9f1c-44c4844d7484 None admin] RBAC: Adding query filter params (role.id=bce78c7e0a534135a820de7f3588aa4d, scope.project.id=fde7ac3ed472433ab3bba56e5e091510, user.id=c88202614775418ba94f478f03431a9b) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:24.420340 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c04af-950f-44fd-9f1c-44c4844d7484 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:24.422362 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c04af-950f-44fd-9f1c-44c4844d7484 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:24.436960 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 345/676] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:35:24 2022] GET /identity/v3/role_assignments?user.id=c88202614775418ba94f478f03431a9b&scope.project.id=fde7ac3ed472433ab3bba56e5e091510&role.id=bce78c7e0a534135a820de7f3588aa4d => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:24.436960 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c04af-950f-44fd-9f1c-44c4844d7484 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:24.483081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21bab7-44fc-4e03-a37b-5a4b4b74c815 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:24.529086 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17efce-a9ac-416b-90b3-b29792f294a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:24.529733 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17efce-a9ac-416b-90b3-b29792f294a1 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:24.530011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17efce-a9ac-416b-90b3-b29792f294a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:24.532905 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17efce-a9ac-416b-90b3-b29792f294a1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:24.544758 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17efce-a9ac-416b-90b3-b29792f294a1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:24.546185 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:24.546372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17efce-a9ac-416b-90b3-b29792f294a1 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:24.559953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-1f17efce-a9ac-416b-90b3-b29792f294a1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 00:35:24.559953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:24.559953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:24.559953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:24.559953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:24.559953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:24.559953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:24.559953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:24.559953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:24.559953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:24.559953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:24.559953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:24.559953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:35:24.559953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:35:24.559953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:35:24.559953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:35:24.567016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:24.567016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:24.567016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:24.567016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:24.567016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:24.567016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:24.567016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:24.567016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:24.567016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:24.567016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:24.567016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:24.567016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:24.567016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:24.567016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:24.567016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:24.567016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:24.567016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:24.567016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:24.567986 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:24.567986 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:24.567986 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:35:24.567986 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:35:24.567986 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:35:24.567986 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:35:24.567986 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:35:24.567986 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:35:24.567986 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 00:35:24.567986 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:24.567986 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 332/677] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:35:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:24.567986 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17efce-a9ac-416b-90b3-b29792f294a1 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:24.573044 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c04af-950f-44fd-9f1c-44c4844d7484 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:24.616510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70450c22-e756-4d51-8d76-27ce2e2a3c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:24.618273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70450c22-e756-4d51-8d76-27ce2e2a3c11 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:24.618960 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70450c22-e756-4d51-8d76-27ce2e2a3c11 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:24.619493 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70450c22-e756-4d51-8d76-27ce2e2a3c11 None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:24.620363 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70450c22-e756-4d51-8d76-27ce2e2a3c11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:24.620888 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70450c22-e756-4d51-8d76-27ce2e2a3c11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:24.622412 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70450c22-e756-4d51-8d76-27ce2e2a3c11 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:24.636084 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 346/678] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:24.636860 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70450c22-e756-4d51-8d76-27ce2e2a3c11 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:24.717276 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17efce-a9ac-416b-90b3-b29792f294a1 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:24.730229 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:24.762494 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ef498-d323-4b49-aff2-2bb6779f30cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:24.763516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ef498-d323-4b49-aff2-2bb6779f30cd None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:24.763968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ef498-d323-4b49-aff2-2bb6779f30cd None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:24.764382 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ef498-d323-4b49-aff2-2bb6779f30cd None admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:24.765704 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8ef498-d323-4b49-aff2-2bb6779f30cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:24.766013 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8ef498-d323-4b49-aff2-2bb6779f30cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:24.766466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70450c22-e756-4d51-8d76-27ce2e2a3c11 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:24.767450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8ef498-d323-4b49-aff2-2bb6779f30cd None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:24.785265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 333/679] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:35:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:24.786379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ef498-d323-4b49-aff2-2bb6779f30cd None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:24.812232 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b8ce3-026c-4967-95ff-1f15bb7b5a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:24.812883 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b8ce3-026c-4967-95ff-1f15bb7b5a43 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:24.813157 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b8ce3-026c-4967-95ff-1f15bb7b5a43 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:24.813422 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b8ce3-026c-4967-95ff-1f15bb7b5a43 None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:24.814616 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b8ce3-026c-4967-95ff-1f15bb7b5a43 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:24.815066 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b8ce3-026c-4967-95ff-1f15bb7b5a43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:24.816860 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b8ce3-026c-4967-95ff-1f15bb7b5a43 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:24.839271 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 347/680] 10.222.0.59 () {56 vars in 1135 bytes} [Fri Aug 26 00:35:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:24.840453 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b8ce3-026c-4967-95ff-1f15bb7b5a43 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:24.846411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:24.916487 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ef498-d323-4b49-aff2-2bb6779f30cd None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:24.962205 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d95399-263b-41d4-9a7d-3a5251954a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:24.962840 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d95399-263b-41d4-9a7d-3a5251954a17 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:24.963137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d95399-263b-41d4-9a7d-3a5251954a17 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:24.963405 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d95399-263b-41d4-9a7d-3a5251954a17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:24.966084 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b8ce3-026c-4967-95ff-1f15bb7b5a43 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:24.975294 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d95399-263b-41d4-9a7d-3a5251954a17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:24.976686 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d95399-263b-41d4-9a7d-3a5251954a17 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:24.988392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-c6d95399-263b-41d4-9a7d-3a5251954a17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:24.988392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:24.988392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:24.988392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:24.988392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:24.988392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:24.988392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:24.988392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:24.988392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:24.988392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:24.988392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:24.988392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:24.988392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:35:24.988392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:35:24.988392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:35:24.990576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:35:24.990576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:24.990576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:24.990576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:24.990576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:24.990576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:24.990576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:24.990576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:24.990576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:24.990576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:24.990576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:24.990576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:24.990576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:24.990576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:24.990576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:24.990576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:24.990576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:24.992851 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:35:24.992851 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:35:24.992851 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:35:24.992851 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:35:24.992851 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:35:24.992851 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:35:24.992851 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:24.992851 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:24.992851 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 334/681] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:24.992851 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d95399-263b-41d4-9a7d-3a5251954a17 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:25.013710 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46773ee-b474-4117-8f0b-e439ba47f831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:25.014357 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46773ee-b474-4117-8f0b-e439ba47f831 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:25.014641 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46773ee-b474-4117-8f0b-e439ba47f831 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:25.028773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46773ee-b474-4117-8f0b-e439ba47f831 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:25.028773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46773ee-b474-4117-8f0b-e439ba47f831 None admin] RBAC: Adding query filter params (role.id=bce78c7e0a534135a820de7f3588aa4d, scope.project.id=fde7ac3ed472433ab3bba56e5e091510, user.id=abed7ebb0aae4e4693ce08e51a149312) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:25.028773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46773ee-b474-4117-8f0b-e439ba47f831 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:25.028773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46773ee-b474-4117-8f0b-e439ba47f831 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:25.029113 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:25.029194 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 348/682] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:35:24 2022] GET /identity/v3/role_assignments?user.id=abed7ebb0aae4e4693ce08e51a149312&scope.project.id=fde7ac3ed472433ab3bba56e5e091510&role.id=bce78c7e0a534135a820de7f3588aa4d => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:25.029194 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46773ee-b474-4117-8f0b-e439ba47f831 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:25.111767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d95399-263b-41d4-9a7d-3a5251954a17 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:25.118955 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:25.157894 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46773ee-b474-4117-8f0b-e439ba47f831 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:25.164151 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914299b0-42d1-44a4-a850-226a256c222b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:25.165104 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914299b0-42d1-44a4-a850-226a256c222b None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:25.165535 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914299b0-42d1-44a4-a850-226a256c222b None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:25.165969 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914299b0-42d1-44a4-a850-226a256c222b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:25.201945 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43036d59-d37b-44cb-a846-dc3fb96fddcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:25.202711 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43036d59-d37b-44cb-a846-dc3fb96fddcd None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:25.203076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43036d59-d37b-44cb-a846-dc3fb96fddcd None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:25.203408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43036d59-d37b-44cb-a846-dc3fb96fddcd None admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:25.204171 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43036d59-d37b-44cb-a846-dc3fb96fddcd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:25.204595 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43036d59-d37b-44cb-a846-dc3fb96fddcd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:25.205986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43036d59-d37b-44cb-a846-dc3fb96fddcd None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:25.222557 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 349/683] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:35:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:25.223800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0f8b3-f13a-442a-85c0-c3cec864f065 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:25.224085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0f8b3-f13a-442a-85c0-c3cec864f065 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:25.224392 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0f8b3-f13a-442a-85c0-c3cec864f065 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:25.225863 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 350/684] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:25.230369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0f8b3-f13a-442a-85c0-c3cec864f065 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:25.255778 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914299b0-42d1-44a4-a850-226a256c222b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:25.257280 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914299b0-42d1-44a4-a850-226a256c222b None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:25.295827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:25.368954 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0f8b3-f13a-442a-85c0-c3cec864f065 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:25.382285 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 335/685] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:35:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:35:25.383581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e72f2d-3b21-429c-9fe8-6bc944980d18 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:25.384231 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e72f2d-3b21-429c-9fe8-6bc944980d18 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:25.384397 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e72f2d-3b21-429c-9fe8-6bc944980d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:25.404313 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-b7e72f2d-3b21-429c-9fe8-6bc944980d18 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:25.406178 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-b7e72f2d-3b21-429c-9fe8-6bc944980d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:25.412869 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:25.417736 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef70503-872e-402f-84a1-f1c6818cfe83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:25.419149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef70503-872e-402f-84a1-f1c6818cfe83 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:25.440766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef70503-872e-402f-84a1-f1c6818cfe83 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:25.440766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef70503-872e-402f-84a1-f1c6818cfe83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:25.440766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef70503-872e-402f-84a1-f1c6818cfe83 None admin] RBAC: Adding query filter params (role.id=bce78c7e0a534135a820de7f3588aa4d, scope.project.id=fde7ac3ed472433ab3bba56e5e091510, user.id=4698a49ed4a04f58a488359a2742e5da) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:25.440766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef70503-872e-402f-84a1-f1c6818cfe83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:25.440766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef70503-872e-402f-84a1-f1c6818cfe83 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:25.441187 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 351/686] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:35:25 2022] GET /identity/v3/role_assignments?user.id=4698a49ed4a04f58a488359a2742e5da&scope.project.id=fde7ac3ed472433ab3bba56e5e091510&role.id=bce78c7e0a534135a820de7f3588aa4d => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:25.441187 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef70503-872e-402f-84a1-f1c6818cfe83 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:25.546944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 336/687] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:35:25.630743 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef70503-872e-402f-84a1-f1c6818cfe83 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:25.669556 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e178d6-efbe-4035-89b5-a3a05aa38cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:25.670251 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e178d6-efbe-4035-89b5-a3a05aa38cb6 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:25.670539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e178d6-efbe-4035-89b5-a3a05aa38cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:25.670819 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e178d6-efbe-4035-89b5-a3a05aa38cb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:25.682351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e178d6-efbe-4035-89b5-a3a05aa38cb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:25.683821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e178d6-efbe-4035-89b5-a3a05aa38cb6 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:25.698617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-b2e178d6-efbe-4035-89b5-a3a05aa38cb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:25.698617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:25.698617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:25.698617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:25.698617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:25.698617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:25.698617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:25.698617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:25.698617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:25.698617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:25.698617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:25.698617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:25.698617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:25.698617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:25.698617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:25.698617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:25.699648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:25.699648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:25.699648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:25.699648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:25.699648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:25.699648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:25.699648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:25.699648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:25.699648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:25.699648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:25.699648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:25.699648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:25.699648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:25.699648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:25.699648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:25.699648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:25.699648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:25.701253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:25.701253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:25.701253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:25.701253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 352/688] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:25.705785 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e72f2d-3b21-429c-9fe8-6bc944980d18 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:25.774809 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:25.818733 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e72f2d-3b21-429c-9fe8-6bc944980d18 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:25.859969 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773728c-172b-49d7-84b6-ad2e9185749f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:25.860990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773728c-172b-49d7-84b6-ad2e9185749f None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:25.861554 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773728c-172b-49d7-84b6-ad2e9185749f None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:25.862032 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773728c-172b-49d7-84b6-ad2e9185749f None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:25.862998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e773728c-172b-49d7-84b6-ad2e9185749f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:25.863508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e773728c-172b-49d7-84b6-ad2e9185749f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:25.865225 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e773728c-172b-49d7-84b6-ad2e9185749f None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:25.878520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e178d6-efbe-4035-89b5-a3a05aa38cb6 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:25.879661 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 337/689] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:25.884803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773728c-172b-49d7-84b6-ad2e9185749f None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:25.940407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:26.005415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773728c-172b-49d7-84b6-ad2e9185749f None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:26.025536 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:26.054729 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a2d90-8c2e-465e-9e4d-2f490e3d39a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:26.055780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a2d90-8c2e-465e-9e4d-2f490e3d39a6 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:26.056293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a2d90-8c2e-465e-9e4d-2f490e3d39a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:26.056790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a2d90-8c2e-465e-9e4d-2f490e3d39a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:26.057316 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e178d6-efbe-4035-89b5-a3a05aa38cb6 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:26.068653 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63a2d90-8c2e-465e-9e4d-2f490e3d39a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:26.070856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63a2d90-8c2e-465e-9e4d-2f490e3d39a6 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:26.094677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-d63a2d90-8c2e-465e-9e4d-2f490e3d39a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:26.094677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:26.094677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:26.094677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:26.094677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:26.094677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:26.094677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:26.094677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:26.094677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:26.094677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:26.094677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:26.094677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:26.094677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:35:26.094677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:35:26.094677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:35:26.095657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:35:26.095657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:26.095657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:26.095657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:26.095657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:26.095657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:26.095657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:26.095657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:26.095657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:26.095657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:26.095657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:26.095657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:26.095657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:26.095657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:26.095657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:26.095657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:26.095657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:26.096617 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:35:26.096617 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:35:26.096617 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:35:26.096617 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:35:26.096617 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:35:26.096617 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:35:26.096617 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:26.096617 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:26.096617 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 338/690] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:26.105409 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a2d90-8c2e-465e-9e4d-2f490e3d39a6 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:26.120541 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a519ce-96f5-4df2-a2b3-2e0be3dd7ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:26.121182 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a519ce-96f5-4df2-a2b3-2e0be3dd7ad2 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:26.121462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a519ce-96f5-4df2-a2b3-2e0be3dd7ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:26.121732 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a519ce-96f5-4df2-a2b3-2e0be3dd7ad2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:26.134300 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a519ce-96f5-4df2-a2b3-2e0be3dd7ad2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:26.139429 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-e4a519ce-96f5-4df2-a2b3-2e0be3dd7ad2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:35:26.139429 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:26.139429 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:26.139429 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:26.139429 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:26.139429 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:26.139429 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:26.139429 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:26.139429 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:26.139429 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:26.139429 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:26.139429 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:26.139429 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:26.139429 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:26.139429 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:26.139429 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:26.140502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:26.140502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:26.140502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:26.140502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:26.140502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:26.140502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:26.140502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:26.140502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:26.140502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:26.140502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:26.140502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:26.140502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:26.140502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:26.140502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:26.140502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:26.140502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:26.141392 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:26.141392 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:26.141392 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:26.141392 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:26.141392 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:26.141392 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:26.141392 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:26.141392 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:26.141392 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:26.141392 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:26.141392 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:35:26.141392 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:26.152497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 353/691] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:35:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:26.152497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a519ce-96f5-4df2-a2b3-2e0be3dd7ad2 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:26.242949 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a2d90-8c2e-465e-9e4d-2f490e3d39a6 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:26.254249 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a519ce-96f5-4df2-a2b3-2e0be3dd7ad2 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:26.256692 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:26.292502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0482fe3-e491-4bff-8119-67cd878dd7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:26.292502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0482fe3-e491-4bff-8119-67cd878dd7cc None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:26.292502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0482fe3-e491-4bff-8119-67cd878dd7cc None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:26.292502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0482fe3-e491-4bff-8119-67cd878dd7cc None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:26.292932 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0482fe3-e491-4bff-8119-67cd878dd7cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:26.293194 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0482fe3-e491-4bff-8119-67cd878dd7cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:26.294742 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0482fe3-e491-4bff-8119-67cd878dd7cc None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:26.298660 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b07d5d-68c1-42f4-b354-b239bf13905d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:26.299569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b07d5d-68c1-42f4-b354-b239bf13905d None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:26.300068 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b07d5d-68c1-42f4-b354-b239bf13905d None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:26.300566 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b07d5d-68c1-42f4-b354-b239bf13905d None admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:26.301482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b07d5d-68c1-42f4-b354-b239bf13905d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:26.301991 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b07d5d-68c1-42f4-b354-b239bf13905d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:26.303661 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b07d5d-68c1-42f4-b354-b239bf13905d None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:26.316495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 354/692] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:35:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:35:26.316495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0482fe3-e491-4bff-8119-67cd878dd7cc None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:26.325376 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 339/693] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:35:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:26.331246 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b07d5d-68c1-42f4-b354-b239bf13905d None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:26.388781 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:26.450644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0482fe3-e491-4bff-8119-67cd878dd7cc None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:26.488254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe868d6-fde6-4427-8667-5ac9ec84d22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:26.489342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe868d6-fde6-4427-8667-5ac9ec84d22a None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:26.489832 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe868d6-fde6-4427-8667-5ac9ec84d22a None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:26.490326 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe868d6-fde6-4427-8667-5ac9ec84d22a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:26.500826 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b07d5d-68c1-42f4-b354-b239bf13905d None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:26.566652 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6923fd1-9fa4-4cd8-81d2-04b0b3206845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:26.567288 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6923fd1-9fa4-4cd8-81d2-04b0b3206845 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:26.567571 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6923fd1-9fa4-4cd8-81d2-04b0b3206845 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:26.567847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6923fd1-9fa4-4cd8-81d2-04b0b3206845 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:26.568625 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6923fd1-9fa4-4cd8-81d2-04b0b3206845 None admin] RBAC: Adding query filter params (role.id=bce78c7e0a534135a820de7f3588aa4d, scope.project.id=fde7ac3ed472433ab3bba56e5e091510, user.id=2f700502671446c797a566f07398a9fc) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:26.568939 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6923fd1-9fa4-4cd8-81d2-04b0b3206845 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:26.570523 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6923fd1-9fa4-4cd8-81d2-04b0b3206845 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:26.585180 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 340/694] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:35:26 2022] GET /identity/v3/role_assignments?user.id=2f700502671446c797a566f07398a9fc&scope.project.id=fde7ac3ed472433ab3bba56e5e091510&role.id=bce78c7e0a534135a820de7f3588aa4d => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:35:26.621597 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe868d6-fde6-4427-8667-5ac9ec84d22a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:26.623539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe868d6-fde6-4427-8667-5ac9ec84d22a None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:26.641398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-dbe868d6-fde6-4427-8667-5ac9ec84d22a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:26.641398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:26.641398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:26.641398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:26.641398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:26.641398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:26.641398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:26.641398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:26.641398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:26.641398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:26.641398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:26.641398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:26.641398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:26.641398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:26.641398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:26.641398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:26.642438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:26.642438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:26.642438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:26.642438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:26.642438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:26.642438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:26.642438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:26.642438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:26.642438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:26.642438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:26.642438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:26.642438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:26.642438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:26.642438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:26.642438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:26.642438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:26.642438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:26.643344 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:26.643344 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:26.643344 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:26.650157 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 355/695] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:26.651895 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6923fd1-9fa4-4cd8-81d2-04b0b3206845 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:26.787403 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6923fd1-9fa4-4cd8-81d2-04b0b3206845 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:26.791959 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:26.832028 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b37f38-ab96-4237-bb1c-ad2b7be1a1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:26.832672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b37f38-ab96-4237-bb1c-ad2b7be1a1a6 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:26.832951 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b37f38-ab96-4237-bb1c-ad2b7be1a1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:26.833219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b37f38-ab96-4237-bb1c-ad2b7be1a1a6 None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:26.833861 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b37f38-ab96-4237-bb1c-ad2b7be1a1a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:26.834159 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b37f38-ab96-4237-bb1c-ad2b7be1a1a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:26.835581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b37f38-ab96-4237-bb1c-ad2b7be1a1a6 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:26.847476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 341/696] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:26.853044 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe868d6-fde6-4427-8667-5ac9ec84d22a None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:26.868448 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c00da7-5695-4d44-933f-d7d2ec3598c9 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:26.869656 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c00da7-5695-4d44-933f-d7d2ec3598c9 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:26.870089 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c00da7-5695-4d44-933f-d7d2ec3598c9 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:26.873890 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 342/697] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:26.879629 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:26.881504 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b735a-1de2-41fb-9ceb-9f0432b6b7ea None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:26.881967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b735a-1de2-41fb-9ceb-9f0432b6b7ea None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:26.882388 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b735a-1de2-41fb-9ceb-9f0432b6b7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:26.907468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-f01b735a-1de2-41fb-9ceb-9f0432b6b7ea None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:26.909603 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-f01b735a-1de2-41fb-9ceb-9f0432b6b7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:26.975690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe868d6-fde6-4427-8667-5ac9ec84d22a None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:26.993419 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 343/698] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:35:27.017507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1688381-a238-48bb-8592-4b27727e0a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:27.018386 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1688381-a238-48bb-8592-4b27727e0a96 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:27.018836 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1688381-a238-48bb-8592-4b27727e0a96 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:27.019253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1688381-a238-48bb-8592-4b27727e0a96 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:27.026995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1688381-a238-48bb-8592-4b27727e0a96 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:27.029013 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1688381-a238-48bb-8592-4b27727e0a96 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:27.041783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-d1688381-a238-48bb-8592-4b27727e0a96 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 00:35:27.041783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:27.041783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:27.041783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:27.041783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:27.041783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:27.041783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:27.041783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:27.041783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:27.041783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:27.041783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:27.041783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:27.041783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:35:27.041783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:35:27.041783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:35:27.041783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:35:27.042997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:27.042997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:27.042997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:27.042997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:27.042997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:27.042997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:27.042997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:27.042997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:27.042997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:27.042997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:27.042997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:27.042997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:27.042997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:27.042997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:27.042997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:27.042997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:27.042997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:27.042997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:27.044196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:27.044196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:27.044196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:35:27.044196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:35:27.044196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:35:27.044196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:35:27.044196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:35:27.044196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:35:27.044196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 00:35:27.044196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:27.050762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 356/699] 10.222.0.59 () {56 vars in 1093 bytes} [Fri Aug 26 00:35:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:27.052285 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01b735a-1de2-41fb-9ceb-9f0432b6b7ea None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:27.162029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:27.170149 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01b735a-1de2-41fb-9ceb-9f0432b6b7ea None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:27.210868 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d9956-c4a4-4f2a-b710-bccd3b042490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:27.211489 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d9956-c4a4-4f2a-b710-bccd3b042490 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:27.211770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d9956-c4a4-4f2a-b710-bccd3b042490 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:27.212042 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d9956-c4a4-4f2a-b710-bccd3b042490 None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:27.212796 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151d9956-c4a4-4f2a-b710-bccd3b042490 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:27.213101 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151d9956-c4a4-4f2a-b710-bccd3b042490 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:27.214577 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151d9956-c4a4-4f2a-b710-bccd3b042490 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:27.240861 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 344/700] 10.222.0.59 () {56 vars in 1133 bytes} [Fri Aug 26 00:35:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:27.246390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1688381-a238-48bb-8592-4b27727e0a96 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:27.325395 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d9956-c4a4-4f2a-b710-bccd3b042490 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:27.398672 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:27.455276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1688381-a238-48bb-8592-4b27727e0a96 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:27.496175 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e387a49-6dac-4310-b9af-1c2b299121a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:27.496820 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e387a49-6dac-4310-b9af-1c2b299121a6 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:27.497097 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e387a49-6dac-4310-b9af-1c2b299121a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:27.497366 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e387a49-6dac-4310-b9af-1c2b299121a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:27.518397 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d9956-c4a4-4f2a-b710-bccd3b042490 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:27.580882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1f5da-db79-49ec-9762-92670e541b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:27.581987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1f5da-db79-49ec-9762-92670e541b4f None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:27.582503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1f5da-db79-49ec-9762-92670e541b4f None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:27.583014 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1f5da-db79-49ec-9762-92670e541b4f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:27.584098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd1f5da-db79-49ec-9762-92670e541b4f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:27.586154 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd1f5da-db79-49ec-9762-92670e541b4f None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:27.600903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-9fd1f5da-db79-49ec-9762-92670e541b4f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 00:35:27.600903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:27.600903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 00:35:27.600903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 00:35:27.600903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 00:35:27.600903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 00:35:27.600903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 00:35:27.600903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:27.600903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 00:35:27.600903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:27.600903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:35:27.600903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:27.600903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:27.600903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:27.600903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:27.600903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:27.600903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:27.601974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:27.601974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:27.601974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:27.601974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:27.601974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:27.601974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:27.601974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 00:35:27.601974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 00:35:27.601974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 00:35:27.601974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 00:35:27.601974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:27.601974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:27.601974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:27.601974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:27.601974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:27.601974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:27.602933 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:27.602933 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:27.602933 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:27.602933 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:27.602933 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:27.602933 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:27.602933 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:27.602933 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:27.602933 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:27.602933 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:27.602933 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 00:35:27.602933 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:27.602933 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 00:35:27.602933 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:27.606902 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 345/701] 10.222.0.59 () {56 vars in 1129 bytes} [Fri Aug 26 00:35:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:27.608251 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995380dd-494c-45a6-aba4-fc6e0b757372 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:27.608556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995380dd-494c-45a6-aba4-fc6e0b757372 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:27.609482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995380dd-494c-45a6-aba4-fc6e0b757372 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:27.609823 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e387a49-6dac-4310-b9af-1c2b299121a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:27.610998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 346/702] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:27.611280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e387a49-6dac-4310-b9af-1c2b299121a6 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:27.611716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995380dd-494c-45a6-aba4-fc6e0b757372 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:27.637836 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:27.704566 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:27.734865 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 357/703] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:35:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 00:35:27.736631 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913aa642-ef28-4cc0-aa2c-8a696ae2107f None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:27.736916 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913aa642-ef28-4cc0-aa2c-8a696ae2107f None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:27.737190 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913aa642-ef28-4cc0-aa2c-8a696ae2107f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:27.738944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995380dd-494c-45a6-aba4-fc6e0b757372 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:27.760124 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-913aa642-ef28-4cc0-aa2c-8a696ae2107f None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:27.761506 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-913aa642-ef28-4cc0-aa2c-8a696ae2107f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:27.779876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7f3d1-3907-4bf9-aeb7-568087c35cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:27.781494 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7f3d1-3907-4bf9-aeb7-568087c35cf1 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:27.781965 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7f3d1-3907-4bf9-aeb7-568087c35cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:27.782419 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7f3d1-3907-4bf9-aeb7-568087c35cf1 None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:27.783234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea7f3d1-3907-4bf9-aeb7-568087c35cf1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:27.783762 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea7f3d1-3907-4bf9-aeb7-568087c35cf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:27.785403 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea7f3d1-3907-4bf9-aeb7-568087c35cf1 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:27.791090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 347/704] 10.222.0.59 () {56 vars in 1124 bytes} [Fri Aug 26 00:35:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:27.792331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7f3d1-3907-4bf9-aeb7-568087c35cf1 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:27.852414 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 358/705] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:35:27.861168 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913aa642-ef28-4cc0-aa2c-8a696ae2107f None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:27.894306 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:27.918482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7f3d1-3907-4bf9-aeb7-568087c35cf1 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:27.954707 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a87ad-5240-4740-b3cc-6c5102c11fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:27.955739 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a87ad-5240-4740-b3cc-6c5102c11fc9 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:27.956814 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a87ad-5240-4740-b3cc-6c5102c11fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:27.957676 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a87ad-5240-4740-b3cc-6c5102c11fc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:27.963074 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913aa642-ef28-4cc0-aa2c-8a696ae2107f None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:27.970896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5a87ad-5240-4740-b3cc-6c5102c11fc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:27.972780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5a87ad-5240-4740-b3cc-6c5102c11fc9 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:27.973359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:27.989447 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-ad5a87ad-5240-4740-b3cc-6c5102c11fc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:27.989447 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:27.989447 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:27.989447 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:27.989447 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:27.989447 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:27.989447 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:27.989447 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:27.989447 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:27.989447 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:27.989447 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:27.989447 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:27.989447 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:27.989447 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:27.989447 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:27.989447 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:27.990450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:27.990450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:27.990450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:27.990450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:27.990450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:27.990450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:27.990450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:27.990450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:27.990450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:27.990450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:27.990450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:27.990450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:27.990450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:27.990450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:27.990450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:27.990450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:27.990450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:27.991359 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:27.991359 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:27.991359 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:27.997779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 348/706] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:27.999107 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5411eb-5879-48d5-97d8-3fcd7941b442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:27.999689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7a3a8-1dcc-4492-b373-6ffb35e39647 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:27.999825 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5411eb-5879-48d5-97d8-3fcd7941b442 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:28.000012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5411eb-5879-48d5-97d8-3fcd7941b442 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:28.000287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5411eb-5879-48d5-97d8-3fcd7941b442 None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:28.000713 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7a3a8-1dcc-4492-b373-6ffb35e39647 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:28.001020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5411eb-5879-48d5-97d8-3fcd7941b442 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:28.001323 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5411eb-5879-48d5-97d8-3fcd7941b442 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:28.001621 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7a3a8-1dcc-4492-b373-6ffb35e39647 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:28.002733 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5411eb-5879-48d5-97d8-3fcd7941b442 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:28.003866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 349/707] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:28.004979 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7a3a8-1dcc-4492-b373-6ffb35e39647 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:28.013137 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 359/708] 10.222.0.59 () {56 vars in 1124 bytes} [Fri Aug 26 00:35:27 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:28.016371 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ced4d1-45f8-495b-88ed-5fc73eebe6f9 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:28.016371 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ced4d1-45f8-495b-88ed-5fc73eebe6f9 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:28.016371 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ced4d1-45f8-495b-88ed-5fc73eebe6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:28.036008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-83ced4d1-45f8-495b-88ed-5fc73eebe6f9 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:28.037313 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-83ced4d1-45f8-495b-88ed-5fc73eebe6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:28.185936 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7a3a8-1dcc-4492-b373-6ffb35e39647 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:28.187620 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 360/709] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:35:28.188402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ced4d1-45f8-495b-88ed-5fc73eebe6f9 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:28.228443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:28.235486 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089e75c-1617-4e5d-93dd-540339fc8730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:28.236176 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089e75c-1617-4e5d-93dd-540339fc8730 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:28.236552 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089e75c-1617-4e5d-93dd-540339fc8730 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:28.236896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089e75c-1617-4e5d-93dd-540339fc8730 None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:28.237769 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089e75c-1617-4e5d-93dd-540339fc8730 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:28.238136 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089e75c-1617-4e5d-93dd-540339fc8730 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:28.239648 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089e75c-1617-4e5d-93dd-540339fc8730 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:28.258290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 350/710] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:28.259672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089e75c-1617-4e5d-93dd-540339fc8730 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:28.300335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ced4d1-45f8-495b-88ed-5fc73eebe6f9 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:28.302338 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:28.340623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10539db-8cc7-4673-8483-146d9e136186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:28.341782 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10539db-8cc7-4673-8483-146d9e136186 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:28.342282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10539db-8cc7-4673-8483-146d9e136186 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:28.342759 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10539db-8cc7-4673-8483-146d9e136186 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:28.362236 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10539db-8cc7-4673-8483-146d9e136186 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:28.369159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10539db-8cc7-4673-8483-146d9e136186 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:28.395152 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-e10539db-8cc7-4673-8483-146d9e136186 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:28.395152 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:28.395152 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:28.395152 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:28.395152 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:28.395152 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:28.395152 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:28.395152 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:28.395152 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:28.395152 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:28.395152 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:28.395152 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:28.395152 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:28.395152 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:28.395152 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:28.395152 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:28.396169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:28.396169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:28.396169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:28.396169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:28.396169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:28.396169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:28.396169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:28.396169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:28.396169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:28.396169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:28.396169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:28.396169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:28.396169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:28.396169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:28.396169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:28.396169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:28.396169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:28.397112 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:28.397112 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:28.397112 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:28.403067 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 361/711] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:28.412500 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10539db-8cc7-4673-8483-146d9e136186 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:28.433943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089e75c-1617-4e5d-93dd-540339fc8730 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:28.505713 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d2cba4-cc23-437b-a39e-afb91906fddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:28.506984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2cba4-cc23-437b-a39e-afb91906fddc None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:28.507497 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2cba4-cc23-437b-a39e-afb91906fddc None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:28.507980 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2cba4-cc23-437b-a39e-afb91906fddc None admin] PATH_INFO: `/v3/projects/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:28.518073 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d2cba4-cc23-437b-a39e-afb91906fddc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:28.520412 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d2cba4-cc23-437b-a39e-afb91906fddc None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:28.535519 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-79d2cba4-cc23-437b-a39e-afb91906fddc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:28.535519 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:28.535519 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:28.535519 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:28.535519 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:28.535519 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:28.535519 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:28.535519 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:28.535519 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:28.535519 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:28.535519 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:28.535519 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:28.535519 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:35:28.535519 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:35:28.535519 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:35:28.536569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:35:28.536569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:28.536569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:28.536569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:28.536569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:28.536569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:28.536569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:28.536569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:28.536569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:28.536569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:28.536569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:28.536569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:28.536569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:28.536569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:28.536569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:28.536569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:28.536569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:28.537577 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:35:28.537577 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:35:28.537577 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:35:28.537577 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:35:28.537577 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:35:28.537577 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:35:28.537577 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:28.537577 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:28.541181 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 351/712] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:28.542576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d2cba4-cc23-437b-a39e-afb91906fddc None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:28.568519 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10539db-8cc7-4673-8483-146d9e136186 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:28.606212 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:28.612347 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d0c55-dd47-4d2a-b8c1-145d1810cf92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:28.613423 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d0c55-dd47-4d2a-b8c1-145d1810cf92 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:28.613771 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d0c55-dd47-4d2a-b8c1-145d1810cf92 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:28.622591 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d0c55-dd47-4d2a-b8c1-145d1810cf92 None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:28.622591 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d0c55-dd47-4d2a-b8c1-145d1810cf92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:28.622591 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d0c55-dd47-4d2a-b8c1-145d1810cf92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:28.622591 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d0c55-dd47-4d2a-b8c1-145d1810cf92 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:28.636767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 362/713] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:28.638066 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d0c55-dd47-4d2a-b8c1-145d1810cf92 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:28.656964 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:28.666804 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d2cba4-cc23-437b-a39e-afb91906fddc None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:28.703393 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a58a6b-5c6a-459c-baf3-efd5654a3a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:28.704027 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a58a6b-5c6a-459c-baf3-efd5654a3a3f None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:28.704310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a58a6b-5c6a-459c-baf3-efd5654a3a3f None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:28.704907 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a58a6b-5c6a-459c-baf3-efd5654a3a3f None admin] PATH_INFO: `/v3/services/network` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:28.705312 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a58a6b-5c6a-459c-baf3-efd5654a3a3f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:28.706690 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a58a6b-5c6a-459c-baf3-efd5654a3a3f None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:28.716435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-f2a58a6b-5c6a-459c-baf3-efd5654a3a3f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 00:35:28.716435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:28.716435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 00:35:28.716435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 00:35:28.716435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 00:35:28.716435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 00:35:28.716435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 00:35:28.716435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:28.716435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 00:35:28.716435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:28.716435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:35:28.716435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:28.716435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:28.716435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:28.716435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:28.716435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:28.716435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:28.717533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:28.717533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:28.717533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:28.717533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:28.717533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:28.717533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:28.717533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 00:35:28.717533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 00:35:28.717533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 00:35:28.717533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 00:35:28.717533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:28.717533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:28.717533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:28.717533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:28.717533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:28.717533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:28.718966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:28.718966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:28.718966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:28.718966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:28.718966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:28.718966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:28.718966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:28.718966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:28.718966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:28.718966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:28.718966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 00:35:28.718966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:28.718966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 00:35:28.718966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:28.723144 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 352/714] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:28 2022] GET /identity/v3/services/network => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:28.724000 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a58a6b-5c6a-459c-baf3-efd5654a3a3f None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:28.752204 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d0c55-dd47-4d2a-b8c1-145d1810cf92 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:28.788500 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87b2e4-16ed-4f1f-83d3-e9214cd05788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:28.789438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87b2e4-16ed-4f1f-83d3-e9214cd05788 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:28.789978 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87b2e4-16ed-4f1f-83d3-e9214cd05788 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:28.790597 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87b2e4-16ed-4f1f-83d3-e9214cd05788 None admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:28.791564 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87b2e4-16ed-4f1f-83d3-e9214cd05788 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:28.792091 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87b2e4-16ed-4f1f-83d3-e9214cd05788 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:28.793761 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87b2e4-16ed-4f1f-83d3-e9214cd05788 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:28.806273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 363/715] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:35:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:28.807422 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87b2e4-16ed-4f1f-83d3-e9214cd05788 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:28.834382 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a58a6b-5c6a-459c-baf3-efd5654a3a3f None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:28.850508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:28.874118 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4ba13-2c5c-4292-8210-1cb23c9ec8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:28.875670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4ba13-2c5c-4292-8210-1cb23c9ec8c9 None admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:28.876098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4ba13-2c5c-4292-8210-1cb23c9ec8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:28.876586 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4ba13-2c5c-4292-8210-1cb23c9ec8c9 None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:28.877376 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4ba13-2c5c-4292-8210-1cb23c9ec8c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:28.879152 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4ba13-2c5c-4292-8210-1cb23c9ec8c9 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:28.889485 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:28.915182 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87b2e4-16ed-4f1f-83d3-e9214cd05788 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:28.951391 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b277bac0-5087-42fc-8b1d-dfbd71a941f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:28.952004 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277bac0-5087-42fc-8b1d-dfbd71a941f3 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:28.952286 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277bac0-5087-42fc-8b1d-dfbd71a941f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:28.953275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277bac0-5087-42fc-8b1d-dfbd71a941f3 None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:28.953913 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b277bac0-5087-42fc-8b1d-dfbd71a941f3 None admin] RBAC: Adding query filter params (name=network) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:28.954326 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b277bac0-5087-42fc-8b1d-dfbd71a941f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:28.955749 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b277bac0-5087-42fc-8b1d-dfbd71a941f3 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:28.960003 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 353/716] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Aug 26 00:35:28 2022] POST /identity/v3/users => generated 254 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:35:28.961885 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 364/717] 10.222.0.59 () {56 vars in 1112 bytes} [Fri Aug 26 00:35:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:28.961885 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac20e55-4f88-4064-89bd-2712a0588ae8 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:28.961885 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac20e55-4f88-4064-89bd-2712a0588ae8 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:28.964175 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac20e55-4f88-4064-89bd-2712a0588ae8 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:28.964175 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 365/718] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:28.965672 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac20e55-4f88-4064-89bd-2712a0588ae8 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:28.965834 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4ba13-2c5c-4292-8210-1cb23c9ec8c9 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:29.095095 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac20e55-4f88-4064-89bd-2712a0588ae8 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:29.109286 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4ba13-2c5c-4292-8210-1cb23c9ec8c9 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:29.141684 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910c2f6-4130-430d-a205-69cf62329b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:29.143044 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910c2f6-4130-430d-a205-69cf62329b47 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:29.143044 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910c2f6-4130-430d-a205-69cf62329b47 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:29.143044 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910c2f6-4130-430d-a205-69cf62329b47 None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:29.143448 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1910c2f6-4130-430d-a205-69cf62329b47 None admin] RBAC: Adding query filter params (type=network) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:29.143750 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1910c2f6-4130-430d-a205-69cf62329b47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:29.145167 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1910c2f6-4130-430d-a205-69cf62329b47 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:29.149943 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 366/719] 10.222.0.59 () {56 vars in 1112 bytes} [Fri Aug 26 00:35:28 2022] GET /identity/v3/services?type=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:29.151238 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d836702-aa01-4ac5-a70f-01122a047649 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:29.151370 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d836702-aa01-4ac5-a70f-01122a047649 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:29.151633 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d836702-aa01-4ac5-a70f-01122a047649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:29.152368 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6099b2-b827-444a-8cce-bccb8f285a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:29.153173 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6099b2-b827-444a-8cce-bccb8f285a39 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:29.153709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6099b2-b827-444a-8cce-bccb8f285a39 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:29.153983 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6099b2-b827-444a-8cce-bccb8f285a39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:29.154715 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6099b2-b827-444a-8cce-bccb8f285a39 None admin] RBAC: Adding query filter params (role.id=1a33e3c8cb8347feb9498595886cc621, scope.project.id=fde7ac3ed472433ab3bba56e5e091510, user.id=0602593263ca4b05a657d5b7697f932f) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:29.155023 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6099b2-b827-444a-8cce-bccb8f285a39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:29.156626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6099b2-b827-444a-8cce-bccb8f285a39 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:29.163098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 354/720] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:35:28 2022] GET /identity/v3/role_assignments?user.id=0602593263ca4b05a657d5b7697f932f&scope.project.id=fde7ac3ed472433ab3bba56e5e091510&role.id=1a33e3c8cb8347feb9498595886cc621 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:29.169640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:29.185914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-7d836702-aa01-4ac5-a70f-01122a047649 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:29.187722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-7d836702-aa01-4ac5-a70f-01122a047649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:29.270209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 367/721] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:35:29.575808 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6099b2-b827-444a-8cce-bccb8f285a39 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:29.625864 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:29.848681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6099b2-b827-444a-8cce-bccb8f285a39 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:29.900182 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2117dc-6719-4479-aa86-09cbcc33213d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:29.901101 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2117dc-6719-4479-aa86-09cbcc33213d None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:29.901486 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2117dc-6719-4479-aa86-09cbcc33213d None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:29.901853 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2117dc-6719-4479-aa86-09cbcc33213d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:29.916376 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2117dc-6719-4479-aa86-09cbcc33213d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:29.921323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-1a2117dc-6719-4479-aa86-09cbcc33213d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:35:29.921323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:29.921323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:29.921323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:29.921323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:29.921323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:29.921323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:29.921323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:29.921323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:29.921323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:29.921323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:29.921323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:29.921323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:29.921323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:29.921323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:29.921323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:29.922275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:29.922275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:29.922275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:29.922275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:29.922275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:29.922275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:29.922275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:29.922275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:29.922275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:29.922275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:29.922275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:29.922275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:29.922275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:29.922275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:29.922275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:29.922275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:29.923547 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:29.923547 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:29.923547 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:29.923547 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:29.923547 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:29.923547 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:29.923547 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:29.923547 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:29.923547 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:29.923547 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:29.923547 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:35:29.923547 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:29.923547 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 355/722] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:35:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:29.928425 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d836702-aa01-4ac5-a70f-01122a047649 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:30.063364 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d836702-aa01-4ac5-a70f-01122a047649 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:30.094623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:30.105314 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d6620-20b9-403c-b781-0eff589d1363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:30.105868 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d6620-20b9-403c-b781-0eff589d1363 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:30.106145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d6620-20b9-403c-b781-0eff589d1363 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:30.106415 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d6620-20b9-403c-b781-0eff589d1363 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:30.107078 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d6620-20b9-403c-b781-0eff589d1363 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:30.107380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d6620-20b9-403c-b781-0eff589d1363 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:30.109281 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d6620-20b9-403c-b781-0eff589d1363 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:30.117573 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 368/723] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:35:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:30.123291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2117dc-6719-4479-aa86-09cbcc33213d None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:30.237720 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:30.246911 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2117dc-6719-4479-aa86-09cbcc33213d None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:30.282444 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b87f94-6a14-4242-a445-ce09e403e571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:30.283075 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b87f94-6a14-4242-a445-ce09e403e571 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:30.283350 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b87f94-6a14-4242-a445-ce09e403e571 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:30.283616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b87f94-6a14-4242-a445-ce09e403e571 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:30.293750 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b87f94-6a14-4242-a445-ce09e403e571 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:30.295211 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b87f94-6a14-4242-a445-ce09e403e571 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:30.310105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-97b87f94-6a14-4242-a445-ce09e403e571 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:30.310105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:30.310105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:30.310105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:30.310105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:30.310105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:30.310105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:30.310105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:30.310105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:30.310105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:30.310105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:30.310105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:30.310105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:30.310105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:30.310105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:30.310105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:30.311137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:30.311137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:30.311137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:30.311137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:30.311137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:30.311137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:30.311137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:30.311137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:30.311137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:30.311137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:30.311137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:30.311137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:30.311137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:30.311137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:30.311137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:30.311137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:30.311137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:30.313179 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:30.313179 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:30.313179 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:30.313179 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 356/724] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:30.321597 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d6620-20b9-403c-b781-0eff589d1363 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:30.437166 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553bdab-3ef6-4f0e-98db-5d932b6d97a5 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:30.437409 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553bdab-3ef6-4f0e-98db-5d932b6d97a5 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:30.437798 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553bdab-3ef6-4f0e-98db-5d932b6d97a5 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:30.440394 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 357/725] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:30.444384 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3fc1b-c219-4d40-b805-6289e4efa309 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:30.444772 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3fc1b-c219-4d40-b805-6289e4efa309 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:30.445116 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3fc1b-c219-4d40-b805-6289e4efa309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:30.463038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-dbd3fc1b-c219-4d40-b805-6289e4efa309 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:30.464307 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-dbd3fc1b-c219-4d40-b805-6289e4efa309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:30.507775 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d6620-20b9-403c-b781-0eff589d1363 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:30.544602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50dd44-cfec-4c88-b613-bcd063096a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:30.545311 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50dd44-cfec-4c88-b613-bcd063096a90 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:30.545654 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50dd44-cfec-4c88-b613-bcd063096a90 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:30.545997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50dd44-cfec-4c88-b613-bcd063096a90 None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:30.547278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50dd44-cfec-4c88-b613-bcd063096a90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:30.547647 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50dd44-cfec-4c88-b613-bcd063096a90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:30.549817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50dd44-cfec-4c88-b613-bcd063096a90 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:30.572796 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 369/726] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:30.574913 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 358/727] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:35:30.575836 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50dd44-cfec-4c88-b613-bcd063096a90 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:30.651819 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:30.696975 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50dd44-cfec-4c88-b613-bcd063096a90 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:30.730940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581bf396-24ba-482d-b6d1-81865330aa62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:30.731573 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bf396-24ba-482d-b6d1-81865330aa62 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:30.731849 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bf396-24ba-482d-b6d1-81865330aa62 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:30.732118 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bf396-24ba-482d-b6d1-81865330aa62 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:30.740125 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581bf396-24ba-482d-b6d1-81865330aa62 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:30.741637 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581bf396-24ba-482d-b6d1-81865330aa62 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:30.752652 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-581bf396-24ba-482d-b6d1-81865330aa62 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 00:35:30.752652 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:30.752652 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:30.752652 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:30.752652 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:30.752652 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:30.752652 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:30.752652 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:30.752652 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:30.752652 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:30.752652 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:30.752652 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:30.752652 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:35:30.752652 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:35:30.752652 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:35:30.752652 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:35:30.753613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:30.753613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:30.753613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:30.753613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:30.753613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:30.753613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:30.753613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:30.753613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:30.753613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:30.753613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:30.753613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:30.753613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:30.753613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:30.753613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:30.753613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:30.753613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:30.753613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:30.753613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:30.754570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:30.754570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:30.754570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:35:30.754570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:35:30.754570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:35:30.754570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:35:30.754570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:35:30.754570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:35:30.754570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 00:35:30.754570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:30.754570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 370/728] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:35:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:30.761599 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3fc1b-c219-4d40-b805-6289e4efa309 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:30.839936 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:30.863020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3fc1b-c219-4d40-b805-6289e4efa309 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:30.897057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09d93b-22aa-4969-982f-facfc10f6cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:30.897673 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09d93b-22aa-4969-982f-facfc10f6cfb None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:30.897948 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09d93b-22aa-4969-982f-facfc10f6cfb None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:30.898214 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09d93b-22aa-4969-982f-facfc10f6cfb None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:30.898936 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09d93b-22aa-4969-982f-facfc10f6cfb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:30.899253 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09d93b-22aa-4969-982f-facfc10f6cfb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:30.900666 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09d93b-22aa-4969-982f-facfc10f6cfb None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:30.920042 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 359/729] 10.222.0.59 () {56 vars in 1135 bytes} [Fri Aug 26 00:35:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:30.929570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581bf396-24ba-482d-b6d1-81865330aa62 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:31.052557 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581bf396-24ba-482d-b6d1-81865330aa62 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:31.062130 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09d93b-22aa-4969-982f-facfc10f6cfb None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:31.067750 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:31.095231 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a83fe96-be76-4419-ad90-34fbe155f997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:31.098803 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83fe96-be76-4419-ad90-34fbe155f997 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:31.098803 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83fe96-be76-4419-ad90-34fbe155f997 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:31.098803 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83fe96-be76-4419-ad90-34fbe155f997 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:31.227839 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a83fe96-be76-4419-ad90-34fbe155f997 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:31.229339 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a83fe96-be76-4419-ad90-34fbe155f997 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:31.255244 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09d93b-22aa-4969-982f-facfc10f6cfb None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:31.314472 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9a537-17ac-45b1-915d-09cdaa021aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:31.315088 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9a537-17ac-45b1-915d-09cdaa021aec None admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:31.315364 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9a537-17ac-45b1-915d-09cdaa021aec None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:31.315629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9a537-17ac-45b1-915d-09cdaa021aec None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:31.316270 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9a537-17ac-45b1-915d-09cdaa021aec None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:31.317926 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9a537-17ac-45b1-915d-09cdaa021aec None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:31.322339 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:31.383853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 371/730] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:35:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 00:35:31.389330 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a83fe96-be76-4419-ad90-34fbe155f997 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:31.417874 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:31.430366 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 360/731] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Aug 26 00:35:31 2022] POST /identity/v3/services => generated 251 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:35:31.503018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a83fe96-be76-4419-ad90-34fbe155f997 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:31.554553 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165d227-d752-464b-a4d7-8fd0e9b7fa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:31.555206 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165d227-d752-464b-a4d7-8fd0e9b7fa42 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:31.555487 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165d227-d752-464b-a4d7-8fd0e9b7fa42 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:31.555758 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165d227-d752-464b-a4d7-8fd0e9b7fa42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:31.568539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7165d227-d752-464b-a4d7-8fd0e9b7fa42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:31.569984 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7165d227-d752-464b-a4d7-8fd0e9b7fa42 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:31.585688 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-7165d227-d752-464b-a4d7-8fd0e9b7fa42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:31.585688 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:31.585688 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:31.585688 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:31.585688 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:31.585688 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:31.585688 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:31.585688 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:31.585688 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:31.585688 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:31.585688 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:31.585688 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:31.585688 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:31.585688 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:31.585688 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:31.585688 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:31.586727 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:31.586727 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:31.586727 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:31.586727 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:31.586727 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:31.586727 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:31.586727 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:31.586727 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:31.586727 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:31.586727 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:31.586727 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:31.586727 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:31.586727 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:31.586727 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:31.586727 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:31.586727 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:31.586727 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:31.587649 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:31.587649 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:31.587649 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:31.587649 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 372/732] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:31.593810 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9a537-17ac-45b1-915d-09cdaa021aec None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:31.601073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce0829-8689-4aa6-8d11-12470ae30a7d None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:31.601371 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce0829-8689-4aa6-8d11-12470ae30a7d None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:31.601623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce0829-8689-4aa6-8d11-12470ae30a7d None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:31.603231 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 373/733] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:35:31.608282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70164fea-44bf-4b04-ba8c-d74383ae273c None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:31.609171 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70164fea-44bf-4b04-ba8c-d74383ae273c None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:31.609454 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70164fea-44bf-4b04-ba8c-d74383ae273c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:31.641327 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-70164fea-44bf-4b04-ba8c-d74383ae273c None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:31.643806 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-70164fea-44bf-4b04-ba8c-d74383ae273c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:31.721294 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:31.734142 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 374/734] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:31 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 00:35:31.736056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4b1d2-c351-49ef-a7b5-437859dabe5d None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:31.736056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4b1d2-c351-49ef-a7b5-437859dabe5d None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:31.736056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4b1d2-c351-49ef-a7b5-437859dabe5d None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:31.738762 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9a537-17ac-45b1-915d-09cdaa021aec None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:31.742562 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 375/735] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:31.744135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02e915-0561-4941-9b00-cc2398cb4a91 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:31.744664 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02e915-0561-4941-9b00-cc2398cb4a91 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:31.745149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02e915-0561-4941-9b00-cc2398cb4a91 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:31.747597 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 376/736] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:31.753365 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f256bdd-fd7d-4913-89c1-919ec817e7dc None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:31.754787 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f256bdd-fd7d-4913-89c1-919ec817e7dc None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:31.755153 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f256bdd-fd7d-4913-89c1-919ec817e7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:31.773025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-8f256bdd-fd7d-4913-89c1-919ec817e7dc None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:31.774273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-8f256bdd-fd7d-4913-89c1-919ec817e7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:31.785910 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92724743-6337-454b-9b35-b6662e781d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:31.787430 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92724743-6337-454b-9b35-b6662e781d68 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:31.787921 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92724743-6337-454b-9b35-b6662e781d68 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:31.788571 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92724743-6337-454b-9b35-b6662e781d68 None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:31.789982 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92724743-6337-454b-9b35-b6662e781d68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:31.790684 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92724743-6337-454b-9b35-b6662e781d68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:31.792516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92724743-6337-454b-9b35-b6662e781d68 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:31.808140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 361/737] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:31.809280 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6bf82-bf80-4484-8666-b50c149c0a2e None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:31.809569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6bf82-bf80-4484-8666-b50c149c0a2e None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:31.809845 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6bf82-bf80-4484-8666-b50c149c0a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:31.828091 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-edf6bf82-bf80-4484-8666-b50c149c0a2e None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:31.829727 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-edf6bf82-bf80-4484-8666-b50c149c0a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:31.847803 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 377/738] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:31 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:35:31.849066 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f256bdd-fd7d-4913-89c1-919ec817e7dc None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:31.882240 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:31.923652 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 362/739] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:31 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 00:35:31.976430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f256bdd-fd7d-4913-89c1-919ec817e7dc None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:32.015927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:32.036830 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5172f-7678-4302-b302-91b9554aac0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:32.037473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5172f-7678-4302-b302-91b9554aac0c None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:32.037758 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5172f-7678-4302-b302-91b9554aac0c None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:32.038031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5172f-7678-4302-b302-91b9554aac0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:32.049754 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b5172f-7678-4302-b302-91b9554aac0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:32.051207 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b5172f-7678-4302-b302-91b9554aac0c None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:32.066953 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-15b5172f-7678-4302-b302-91b9554aac0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:32.066953 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:32.066953 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:32.066953 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:32.066953 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:32.066953 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:32.066953 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:32.066953 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:32.066953 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:32.066953 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:32.066953 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:32.066953 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:32.066953 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:35:32.066953 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:35:32.066953 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:35:32.068696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:35:32.068696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:32.068696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:32.068696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:32.068696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:32.068696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:32.068696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:32.068696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:32.068696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:32.068696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:32.068696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:32.068696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:32.068696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:32.068696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:32.068696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:32.068696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:32.068696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:32.069943 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:35:32.069943 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:35:32.069943 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:35:32.069943 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:35:32.069943 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:35:32.069943 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:35:32.069943 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:32.069943 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:32.069943 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 378/740] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:32.073959 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf6bf82-bf80-4484-8666-b50c149c0a2e None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:32.130195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5172f-7678-4302-b302-91b9554aac0c None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:32.192990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf6bf82-bf80-4484-8666-b50c149c0a2e None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:32.229122 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5242b5e-e9e9-4dd5-9560-b8dbbb147b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:32.229835 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5242b5e-e9e9-4dd5-9560-b8dbbb147b41 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:32.230219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5242b5e-e9e9-4dd5-9560-b8dbbb147b41 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:32.230502 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5242b5e-e9e9-4dd5-9560-b8dbbb147b41 None admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:32.231205 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5242b5e-e9e9-4dd5-9560-b8dbbb147b41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:32.231514 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5242b5e-e9e9-4dd5-9560-b8dbbb147b41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:32.233173 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5242b5e-e9e9-4dd5-9560-b8dbbb147b41 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:32.239334 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5172f-7678-4302-b302-91b9554aac0c None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:32.246371 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 363/741] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:35:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:32.247198 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5242b5e-e9e9-4dd5-9560-b8dbbb147b41 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:32.275575 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463a695-7dc9-4b71-878d-ed19fef50ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:32.276250 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463a695-7dc9-4b71-878d-ed19fef50ac7 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:32.276554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463a695-7dc9-4b71-878d-ed19fef50ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:32.276828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463a695-7dc9-4b71-878d-ed19fef50ac7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:32.287723 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e463a695-7dc9-4b71-878d-ed19fef50ac7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:32.292654 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-e463a695-7dc9-4b71-878d-ed19fef50ac7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:35:32.292654 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:32.292654 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:32.292654 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:32.292654 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:32.292654 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:32.292654 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:32.292654 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:32.292654 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:32.292654 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:32.292654 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:32.292654 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:32.292654 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:32.292654 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:32.292654 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:32.292654 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:32.294611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:32.294611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:32.294611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:32.294611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:32.294611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:32.294611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:32.294611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:32.294611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:32.294611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:32.294611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:32.294611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:32.294611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:32.294611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:32.294611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:32.294611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:32.294611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:32.295978 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:32.295978 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:32.295978 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:32.295978 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:32.295978 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:32.295978 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:32.295978 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:32.295978 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:32.295978 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:32.295978 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:32.295978 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:35:32.295978 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:32.295978 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 379/742] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:35:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:32.295978 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463a695-7dc9-4b71-878d-ed19fef50ac7 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:32.297766 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:32.343124 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:32.367455 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5242b5e-e9e9-4dd5-9560-b8dbbb147b41 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:32.402201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b996c-4dd6-4f2e-94e4-f046a4712d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:32.402778 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b996c-4dd6-4f2e-94e4-f046a4712d5a None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:32.403057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b996c-4dd6-4f2e-94e4-f046a4712d5a None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:32.403321 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b996c-4dd6-4f2e-94e4-f046a4712d5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:32.407716 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463a695-7dc9-4b71-878d-ed19fef50ac7 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:32.413809 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b996c-4dd6-4f2e-94e4-f046a4712d5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:32.419332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-569b996c-4dd6-4f2e-94e4-f046a4712d5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:32.419332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:32.419332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:32.419332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:32.419332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:32.419332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:32.419332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:32.419332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:32.419332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:32.419332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:32.419332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:32.419332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:32.419332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:32.419332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:32.419332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:32.419332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:32.420314 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:32.420314 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:32.420314 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:32.420314 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:32.420314 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:32.420314 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:32.420314 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:32.420314 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:32.420314 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:32.420314 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:32.420314 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:32.420314 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:32.420314 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:32.420314 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:32.420314 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:32.420314 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:32.421853 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:32.421853 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:32.421853 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:32.421853 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:32.421853 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:32.421853 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:32.421853 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:32.421853 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:32.421853 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:32.421853 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:32.421853 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:32.421853 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:32.424064 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 364/743] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Aug 26 00:35:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:32.425113 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b996c-4dd6-4f2e-94e4-f046a4712d5a None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:32.451583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d633052-8d1c-4bcf-bd11-af88c6ad8cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:32.452295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d633052-8d1c-4bcf-bd11-af88c6ad8cb2 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:32.452680 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d633052-8d1c-4bcf-bd11-af88c6ad8cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:32.453083 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d633052-8d1c-4bcf-bd11-af88c6ad8cb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:32.453872 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d633052-8d1c-4bcf-bd11-af88c6ad8cb2 None admin] RBAC: Adding query filter params (role.id=1a33e3c8cb8347feb9498595886cc621, scope.project.id=fde7ac3ed472433ab3bba56e5e091510, user.id=2f700502671446c797a566f07398a9fc) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:32.454336 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d633052-8d1c-4bcf-bd11-af88c6ad8cb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:32.455748 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d633052-8d1c-4bcf-bd11-af88c6ad8cb2 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:32.468514 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 380/744] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:35:32 2022] GET /identity/v3/role_assignments?user.id=2f700502671446c797a566f07398a9fc&scope.project.id=fde7ac3ed472433ab3bba56e5e091510&role.id=1a33e3c8cb8347feb9498595886cc621 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:32.468514 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d633052-8d1c-4bcf-bd11-af88c6ad8cb2 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:32.546815 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b996c-4dd6-4f2e-94e4-f046a4712d5a None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:32.550373 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:32.596239 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a7316-2992-41f4-9ff7-59c5c98a5c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:32.608072 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a7316-2992-41f4-9ff7-59c5c98a5c9a None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:32.608072 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a7316-2992-41f4-9ff7-59c5c98a5c9a None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:32.608072 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a7316-2992-41f4-9ff7-59c5c98a5c9a None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:32.608072 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a7316-2992-41f4-9ff7-59c5c98a5c9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:32.608072 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a7316-2992-41f4-9ff7-59c5c98a5c9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:32.609055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d633052-8d1c-4bcf-bd11-af88c6ad8cb2 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:32.609141 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a7316-2992-41f4-9ff7-59c5c98a5c9a None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:32.609917 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:32.614077 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 365/745] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:35:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:32.615134 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a7316-2992-41f4-9ff7-59c5c98a5c9a None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:32.654815 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfabfd1-eadb-4d69-9d5c-98b235d46f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:32.655880 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfabfd1-eadb-4d69-9d5c-98b235d46f4b None admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:32.656178 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfabfd1-eadb-4d69-9d5c-98b235d46f4b None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:32.656454 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfabfd1-eadb-4d69-9d5c-98b235d46f4b None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:32.657338 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfabfd1-eadb-4d69-9d5c-98b235d46f4b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:32.659857 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfabfd1-eadb-4d69-9d5c-98b235d46f4b None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:32.729628 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a7316-2992-41f4-9ff7-59c5c98a5c9a None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:32.744746 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 381/746] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Aug 26 00:35:32 2022] POST /identity/v3/services => generated 240 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:35:32.746045 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfabfd1-eadb-4d69-9d5c-98b235d46f4b None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:32.786522 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72c13f-f03b-4557-aaed-6730e6d1b42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:32.787241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72c13f-f03b-4557-aaed-6730e6d1b42c None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:32.787444 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72c13f-f03b-4557-aaed-6730e6d1b42c None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:32.787767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72c13f-f03b-4557-aaed-6730e6d1b42c None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:32.789030 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e72c13f-f03b-4557-aaed-6730e6d1b42c None admin] RBAC: Adding query filter params (name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:32.789387 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e72c13f-f03b-4557-aaed-6730e6d1b42c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:32.791641 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e72c13f-f03b-4557-aaed-6730e6d1b42c None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:32.802978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 366/747] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Aug 26 00:35:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:32.810712 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72c13f-f03b-4557-aaed-6730e6d1b42c None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:32.863412 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:32.928833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfabfd1-eadb-4d69-9d5c-98b235d46f4b None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:32.975627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3dc9a-8171-4ac2-b26e-8b12c34bac94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:32.976283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3dc9a-8171-4ac2-b26e-8b12c34bac94 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:32.976601 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3dc9a-8171-4ac2-b26e-8b12c34bac94 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:32.976875 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3dc9a-8171-4ac2-b26e-8b12c34bac94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:32.982029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72c13f-f03b-4557-aaed-6730e6d1b42c None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:32.990995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c3dc9a-8171-4ac2-b26e-8b12c34bac94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:32.992684 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c3dc9a-8171-4ac2-b26e-8b12c34bac94 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:33.009683 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-c1c3dc9a-8171-4ac2-b26e-8b12c34bac94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:33.009683 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:33.009683 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:33.009683 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:33.009683 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:33.009683 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:33.009683 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:33.009683 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:33.009683 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:33.009683 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:33.009683 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:33.009683 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:33.009683 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:33.009683 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:33.009683 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:33.009683 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:33.010768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:33.010768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:33.010768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:33.010768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:33.010768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:33.010768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:33.010768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:33.010768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:33.010768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:33.010768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:33.010768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:33.010768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:33.010768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:33.010768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:33.010768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:33.010768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:33.010768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:33.012306 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:33.012306 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:33.012306 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:33.012306 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 382/748] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:33.017359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3dc9a-8171-4ac2-b26e-8b12c34bac94 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:33.023094 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9c05b-a530-4eb2-b205-610f2c435280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:33.025254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9c05b-a530-4eb2-b205-610f2c435280 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:33.025254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9c05b-a530-4eb2-b205-610f2c435280 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:33.025254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9c05b-a530-4eb2-b205-610f2c435280 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:33.036288 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a9c05b-a530-4eb2-b205-610f2c435280 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:33.038072 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a9c05b-a530-4eb2-b205-610f2c435280 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:33.039170 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:33.055011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-a8a9c05b-a530-4eb2-b205-610f2c435280 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:33.055011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:33.055011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:33.055011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:33.055011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:33.055011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:33.055011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:33.055011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:33.055011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:33.055011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:33.055011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:33.055011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:33.055011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:33.055011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:33.055011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:33.055011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:33.056045 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:33.056045 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:33.056045 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:33.056045 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:33.056045 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:33.056045 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:33.056045 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:33.056045 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:33.056045 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:33.056045 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:33.056045 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:33.056045 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:33.056045 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:33.056045 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:33.056045 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:33.056045 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:33.056045 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:33.057005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:33.057005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:33.057005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:33.057733 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 367/749] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:33.063480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9c05b-a530-4eb2-b205-610f2c435280 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:33.147131 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:33.172501 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9c05b-a530-4eb2-b205-610f2c435280 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:33.219347 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c498d-49f6-43c5-9052-b003f207182b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:33.219877 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c498d-49f6-43c5-9052-b003f207182b None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:33.220214 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c498d-49f6-43c5-9052-b003f207182b None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:33.220504 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c498d-49f6-43c5-9052-b003f207182b None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:33.221159 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c498d-49f6-43c5-9052-b003f207182b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:33.221509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c498d-49f6-43c5-9052-b003f207182b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:33.222885 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c498d-49f6-43c5-9052-b003f207182b None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:33.242980 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 368/750] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:35:33.249331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c498d-49f6-43c5-9052-b003f207182b None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:33.321563 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3dc9a-8171-4ac2-b26e-8b12c34bac94 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:33.385759 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b096b0-3091-44b9-89a7-6b63c7f33bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:33.386428 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b096b0-3091-44b9-89a7-6b63c7f33bc4 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:33.386830 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b096b0-3091-44b9-89a7-6b63c7f33bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:33.387207 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b096b0-3091-44b9-89a7-6b63c7f33bc4 None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:33.388028 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b096b0-3091-44b9-89a7-6b63c7f33bc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:33.388333 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b096b0-3091-44b9-89a7-6b63c7f33bc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:33.389785 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b096b0-3091-44b9-89a7-6b63c7f33bc4 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:33.419352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 383/751] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:35:33.425472 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b096b0-3091-44b9-89a7-6b63c7f33bc4 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:33.434918 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c498d-49f6-43c5-9052-b003f207182b None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:33.474555 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6081c989-49bf-46ff-af92-e0dee2c488b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:33.475474 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081c989-49bf-46ff-af92-e0dee2c488b2 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:33.475894 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081c989-49bf-46ff-af92-e0dee2c488b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:33.476316 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081c989-49bf-46ff-af92-e0dee2c488b2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:33.479492 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:33.486133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6081c989-49bf-46ff-af92-e0dee2c488b2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:33.487861 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6081c989-49bf-46ff-af92-e0dee2c488b2 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:33.500529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-6081c989-49bf-46ff-af92-e0dee2c488b2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 00:35:33.500529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:33.500529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:33.500529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:33.500529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:33.500529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:33.500529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:33.500529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:33.500529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:33.500529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:33.500529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:33.500529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:33.500529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:35:33.500529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:35:33.500529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:35:33.500529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:35:33.501736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:33.501736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:33.501736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:33.501736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:33.501736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:33.501736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:33.501736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:33.501736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:33.501736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:33.501736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:33.501736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:33.501736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:33.501736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:33.501736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:33.501736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:33.501736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:33.501736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:33.501736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:33.502900 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:33.502900 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:33.502900 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:35:33.502900 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:35:33.502900 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:35:33.502900 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:35:33.502900 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:35:33.502900 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:35:33.502900 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 00:35:33.502900 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:33.506636 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 369/752] 10.222.0.59 () {56 vars in 1117 bytes} [Fri Aug 26 00:35:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:33.513358 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6081c989-49bf-46ff-af92-e0dee2c488b2 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:33.601191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:33.628320 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6081c989-49bf-46ff-af92-e0dee2c488b2 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:33.632236 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b096b0-3091-44b9-89a7-6b63c7f33bc4 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:33.670306 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05d56b-9de5-4805-bbb5-defd41f15718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:33.671856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05d56b-9de5-4805-bbb5-defd41f15718 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:33.672368 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05d56b-9de5-4805-bbb5-defd41f15718 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:33.672832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05d56b-9de5-4805-bbb5-defd41f15718 None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:33.673704 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d05d56b-9de5-4805-bbb5-defd41f15718 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:33.674149 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d05d56b-9de5-4805-bbb5-defd41f15718 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:33.675819 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d05d56b-9de5-4805-bbb5-defd41f15718 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:33.695194 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62517d08-22af-4fd3-ab00-8880a6630dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:33.696259 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62517d08-22af-4fd3-ab00-8880a6630dc9 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:33.696675 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62517d08-22af-4fd3-ab00-8880a6630dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:33.697058 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62517d08-22af-4fd3-ab00-8880a6630dc9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:33.702412 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 370/753] 10.222.0.59 () {56 vars in 1149 bytes} [Fri Aug 26 00:35:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:33.704834 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd309cb-f5af-42f7-b9d5-ec67240cb81d None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:33.705382 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd309cb-f5af-42f7-b9d5-ec67240cb81d None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:33.705807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd309cb-f5af-42f7-b9d5-ec67240cb81d None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:33.707588 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 371/754] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:33.708587 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd309cb-f5af-42f7-b9d5-ec67240cb81d None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:33.714609 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62517d08-22af-4fd3-ab00-8880a6630dc9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:33.716280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62517d08-22af-4fd3-ab00-8880a6630dc9 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:33.733436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-62517d08-22af-4fd3-ab00-8880a6630dc9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 00:35:33.733436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:33.733436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:33.733436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:33.733436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:33.733436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:33.733436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:33.733436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:33.733436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:33.733436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:33.733436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:33.733436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:33.733436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:35:33.733436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:35:33.733436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:35:33.733436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:35:33.735151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:33.735151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:33.735151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:33.735151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:33.735151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:33.735151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:33.735151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:33.735151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:33.735151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:33.735151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:33.735151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:33.735151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:33.735151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:33.735151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:33.735151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:33.735151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:33.735151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:33.735151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:33.736096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:33.736096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:33.736096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:35:33.736096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:35:33.736096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:35:33.736096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:35:33.736096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:35:33.736096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:35:33.736096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 00:35:33.736096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:33.742145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 384/755] 10.222.0.59 () {56 vars in 1093 bytes} [Fri Aug 26 00:35:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:33.742145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d891e-392b-42bb-9aef-59eae34e3bc1 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:33.742145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d891e-392b-42bb-9aef-59eae34e3bc1 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:33.742145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d891e-392b-42bb-9aef-59eae34e3bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:33.763134 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-953d891e-392b-42bb-9aef-59eae34e3bc1 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:33.765500 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-953d891e-392b-42bb-9aef-59eae34e3bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:33.785323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:33.838757 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 385/756] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:35:33.839925 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d891e-392b-42bb-9aef-59eae34e3bc1 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:33.846938 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd309cb-f5af-42f7-b9d5-ec67240cb81d None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:33.893530 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098fe84c-6b87-437d-a291-176a45f58719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:33.894630 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fe84c-6b87-437d-a291-176a45f58719 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:33.894987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fe84c-6b87-437d-a291-176a45f58719 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:33.895339 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fe84c-6b87-437d-a291-176a45f58719 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:33.913570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:33.959725 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d891e-392b-42bb-9aef-59eae34e3bc1 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:34.000023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6a252-18fb-486d-892f-403fdb6814ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:34.000991 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6a252-18fb-486d-892f-403fdb6814ce None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:34.001445 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6a252-18fb-486d-892f-403fdb6814ce None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:34.001935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6a252-18fb-486d-892f-403fdb6814ce None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:34.002941 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6a252-18fb-486d-892f-403fdb6814ce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:34.003439 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6a252-18fb-486d-892f-403fdb6814ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:34.005061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6a252-18fb-486d-892f-403fdb6814ce None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:34.017804 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098fe84c-6b87-437d-a291-176a45f58719 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:34.019256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098fe84c-6b87-437d-a291-176a45f58719 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:34.026755 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 386/757] 10.222.0.59 () {56 vars in 1133 bytes} [Fri Aug 26 00:35:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:34.033264 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6a252-18fb-486d-892f-403fdb6814ce None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:34.099106 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:34.167174 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 372/758] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:35:33 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 00:35:34.173253 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098fe84c-6b87-437d-a291-176a45f58719 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:34.260694 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6a252-18fb-486d-892f-403fdb6814ce None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:34.279436 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098fe84c-6b87-437d-a291-176a45f58719 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:34.287650 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:34.296345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15440417-47f7-4938-813c-13ad2e47b6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:34.297031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15440417-47f7-4938-813c-13ad2e47b6ba None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:34.297294 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15440417-47f7-4938-813c-13ad2e47b6ba None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:34.300082 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15440417-47f7-4938-813c-13ad2e47b6ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:34.307862 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15440417-47f7-4938-813c-13ad2e47b6ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:34.309404 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15440417-47f7-4938-813c-13ad2e47b6ba None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:34.317458 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ceb77-072a-405b-9807-1bd90733023a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:34.318356 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ceb77-072a-405b-9807-1bd90733023a None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:34.318837 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ceb77-072a-405b-9807-1bd90733023a None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:34.319323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ceb77-072a-405b-9807-1bd90733023a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:34.324434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-15440417-47f7-4938-813c-13ad2e47b6ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:34.324434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:34.324434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:34.324434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:34.324434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:34.324434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:34.324434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:34.324434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:34.324434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:34.324434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:34.324434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:34.324434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:34.324434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:34.324434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:34.324434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:34.324434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:34.325704 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:34.325704 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:34.325704 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:34.325704 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:34.325704 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:34.325704 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:34.325704 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:34.325704 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:34.325704 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:34.325704 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:34.325704 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:34.325704 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:34.325704 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:34.325704 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:34.325704 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:34.325704 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:34.325704 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:34.326597 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:34.326597 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:34.326597 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:34.326597 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 387/759] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:34.326826 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15440417-47f7-4938-813c-13ad2e47b6ba None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:34.338338 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861ceb77-072a-405b-9807-1bd90733023a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:34.340065 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861ceb77-072a-405b-9807-1bd90733023a None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:34.356002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-861ceb77-072a-405b-9807-1bd90733023a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:34.356002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:34.356002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:34.356002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:34.356002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:34.356002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:34.356002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:34.356002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:34.356002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:34.356002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:34.356002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:34.356002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:34.356002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:34.356002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:34.356002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:34.356002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:34.357057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:34.357057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:34.357057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:34.357057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:34.357057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:34.357057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:34.357057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:34.357057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:34.357057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:34.357057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:34.357057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:34.357057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:34.357057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:34.357057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:34.357057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:34.357057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:34.357057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:34.357952 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:34.357952 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:34.357952 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:34.364457 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 373/760] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:34.365536 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ceb77-072a-405b-9807-1bd90733023a None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:34.375255 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:34.434399 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15440417-47f7-4938-813c-13ad2e47b6ba None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:34.470868 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc99e94-aa29-4e8f-8c4f-87924360b9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:34.471505 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc99e94-aa29-4e8f-8c4f-87924360b9f8 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:34.472014 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc99e94-aa29-4e8f-8c4f-87924360b9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:34.472123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc99e94-aa29-4e8f-8c4f-87924360b9f8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:34.472655 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc99e94-aa29-4e8f-8c4f-87924360b9f8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:34.474008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc99e94-aa29-4e8f-8c4f-87924360b9f8 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:34.478376 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ceb77-072a-405b-9807-1bd90733023a None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:34.485738 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-7cc99e94-aa29-4e8f-8c4f-87924360b9f8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 00:35:34.485738 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:34.485738 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 00:35:34.485738 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 00:35:34.485738 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 00:35:34.485738 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 00:35:34.485738 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 00:35:34.485738 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:34.485738 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 00:35:34.485738 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:34.485738 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:35:34.485738 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:34.485738 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:34.485738 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:34.485738 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:34.485738 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:34.485738 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:34.486772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:34.486772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:34.486772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:34.486772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:34.486772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:34.486772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:34.486772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 00:35:34.486772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 00:35:34.486772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 00:35:34.486772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 00:35:34.486772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:34.486772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:34.486772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:34.486772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:34.486772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:34.486772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:34.487618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:34.487618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:34.487618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:34.487618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:34.487618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:34.487618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:34.487618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:34.487618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:34.487618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:34.487618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:34.487618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 00:35:34.487618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:34.487618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 00:35:34.487618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:34.491709 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 388/761] 10.222.0.59 () {56 vars in 1129 bytes} [Fri Aug 26 00:35:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:34.492585 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc99e94-aa29-4e8f-8c4f-87924360b9f8 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:34.518660 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a331d01-e4bb-47e9-9e5f-b3fe5f8f85ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:34.519590 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a331d01-e4bb-47e9-9e5f-b3fe5f8f85ba None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:34.520086 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a331d01-e4bb-47e9-9e5f-b3fe5f8f85ba None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:34.520596 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a331d01-e4bb-47e9-9e5f-b3fe5f8f85ba None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:34.521434 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a331d01-e4bb-47e9-9e5f-b3fe5f8f85ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:34.521974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a331d01-e4bb-47e9-9e5f-b3fe5f8f85ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:34.523561 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a331d01-e4bb-47e9-9e5f-b3fe5f8f85ba None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:34.541519 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:34.541934 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 374/762] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:34.543023 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a331d01-e4bb-47e9-9e5f-b3fe5f8f85ba None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:34.596369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc99e94-aa29-4e8f-8c4f-87924360b9f8 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:34.612607 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:34.634578 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec2def-42e9-47d2-b7d3-3bfb31756a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:34.635244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec2def-42e9-47d2-b7d3-3bfb31756a64 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:34.635466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec2def-42e9-47d2-b7d3-3bfb31756a64 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:34.635740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec2def-42e9-47d2-b7d3-3bfb31756a64 None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:34.636381 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec2def-42e9-47d2-b7d3-3bfb31756a64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:34.636756 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec2def-42e9-47d2-b7d3-3bfb31756a64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:34.638163 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec2def-42e9-47d2-b7d3-3bfb31756a64 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:34.645656 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a331d01-e4bb-47e9-9e5f-b3fe5f8f85ba None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:34.651657 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 389/763] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:34.652809 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec2def-42e9-47d2-b7d3-3bfb31756a64 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:34.682987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673696ba-4f87-457e-9045-1af3f75dd716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:34.683916 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673696ba-4f87-457e-9045-1af3f75dd716 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:34.684378 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673696ba-4f87-457e-9045-1af3f75dd716 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:34.684880 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673696ba-4f87-457e-9045-1af3f75dd716 None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:34.685633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673696ba-4f87-457e-9045-1af3f75dd716 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:34.686012 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673696ba-4f87-457e-9045-1af3f75dd716 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:34.687913 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673696ba-4f87-457e-9045-1af3f75dd716 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:34.696931 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 375/764] 10.222.0.59 () {56 vars in 1124 bytes} [Fri Aug 26 00:35:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:34.698412 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218bfc2-5570-4b61-ad46-a5a8730ce154 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:34.698702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218bfc2-5570-4b61-ad46-a5a8730ce154 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:34.698976 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218bfc2-5570-4b61-ad46-a5a8730ce154 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:34.704208 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 376/765] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:34.704208 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c218bfc2-5570-4b61-ad46-a5a8730ce154 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:34.799388 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec2def-42e9-47d2-b7d3-3bfb31756a64 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:34.804993 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:34.832003 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c218bfc2-5570-4b61-ad46-a5a8730ce154 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:34.843398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51daf352-0445-4d95-a8b1-0d7c879cca76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:34.864872 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51daf352-0445-4d95-a8b1-0d7c879cca76 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:34.864872 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51daf352-0445-4d95-a8b1-0d7c879cca76 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:34.864872 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51daf352-0445-4d95-a8b1-0d7c879cca76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:34.864872 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51daf352-0445-4d95-a8b1-0d7c879cca76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:34.864872 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51daf352-0445-4d95-a8b1-0d7c879cca76 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:34.872495 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:34.877794 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-51daf352-0445-4d95-a8b1-0d7c879cca76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:34.877794 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:34.877794 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:34.877794 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:34.877794 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:34.877794 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:34.877794 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:34.877794 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:34.877794 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:34.877794 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:34.877794 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:34.877794 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:34.877794 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:35:34.877794 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:35:34.877794 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:35:34.879583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:35:34.879583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:34.879583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:34.879583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:34.879583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:34.879583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:34.879583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:34.879583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:34.879583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:34.879583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:34.879583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:34.879583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:34.879583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:34.879583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:34.879583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:34.879583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:34.879583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:34.882254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:35:34.882254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:35:34.882254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:35:34.882254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:35:34.882254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:35:34.882254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:35:34.882254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:34.882254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:34.882254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 390/766] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:34.882254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51daf352-0445-4d95-a8b1-0d7c879cca76 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:34.885954 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4501cd2-e042-47da-b56a-917df2e59974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:34.886630 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4501cd2-e042-47da-b56a-917df2e59974 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:34.886993 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4501cd2-e042-47da-b56a-917df2e59974 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:34.887273 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4501cd2-e042-47da-b56a-917df2e59974 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:34.895834 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4501cd2-e042-47da-b56a-917df2e59974 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:34.897324 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4501cd2-e042-47da-b56a-917df2e59974 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:34.910516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-f4501cd2-e042-47da-b56a-917df2e59974 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:34.910516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:34.910516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:34.910516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:34.910516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:34.910516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:34.910516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:34.910516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:34.910516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:34.910516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:34.910516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:34.910516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:34.910516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:35:34.910516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:35:34.910516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:35:34.911668 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:35:34.911668 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:34.911668 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:34.911668 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:34.911668 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:34.911668 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:34.911668 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:34.911668 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:34.911668 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:34.911668 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:34.911668 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:34.911668 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:34.911668 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:34.911668 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:34.911668 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:34.911668 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:34.911668 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:34.912616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:35:34.912616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:35:34.912616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:35:34.912616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:35:34.912616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:35:34.912616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:35:34.912616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:34.912616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:34.912616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 377/767] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:34.913115 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94088a70-096c-4859-b591-81fee103ffea None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:34.913115 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94088a70-096c-4859-b591-81fee103ffea None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:34.913682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94088a70-096c-4859-b591-81fee103ffea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:34.932973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-94088a70-096c-4859-b591-81fee103ffea None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:34.934153 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-94088a70-096c-4859-b591-81fee103ffea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:35.014672 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51daf352-0445-4d95-a8b1-0d7c879cca76 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:35.028736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 378/768] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:35:35.028736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94088a70-096c-4859-b591-81fee103ffea None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:35.051724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aca89f-a510-41da-917f-23e3cd6c8594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:35.052254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aca89f-a510-41da-917f-23e3cd6c8594 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:35.052571 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aca89f-a510-41da-917f-23e3cd6c8594 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:35.053215 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aca89f-a510-41da-917f-23e3cd6c8594 None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:35.053472 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aca89f-a510-41da-917f-23e3cd6c8594 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:35.053772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aca89f-a510-41da-917f-23e3cd6c8594 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:35.055134 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aca89f-a510-41da-917f-23e3cd6c8594 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:35.064908 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 391/769] 10.222.0.59 () {56 vars in 1124 bytes} [Fri Aug 26 00:35:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:35.064908 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aca89f-a510-41da-917f-23e3cd6c8594 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:35.093309 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:35.137630 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94088a70-096c-4859-b591-81fee103ffea None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:35.159315 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:35.170607 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aca89f-a510-41da-917f-23e3cd6c8594 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:35.175182 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e18b00-e848-4ef2-818c-7d0678d61089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:35.175826 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e18b00-e848-4ef2-818c-7d0678d61089 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:35.176104 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e18b00-e848-4ef2-818c-7d0678d61089 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:35.176375 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e18b00-e848-4ef2-818c-7d0678d61089 None admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:35.177123 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e18b00-e848-4ef2-818c-7d0678d61089 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:35.177428 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e18b00-e848-4ef2-818c-7d0678d61089 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:35.178931 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e18b00-e848-4ef2-818c-7d0678d61089 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:35.191442 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 379/770] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:35:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:35.193521 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab690130-9208-49c1-95c1-18a2d4d14ef8 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:35.193755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab690130-9208-49c1-95c1-18a2d4d14ef8 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:35.194025 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab690130-9208-49c1-95c1-18a2d4d14ef8 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:35.195383 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 380/771] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:35.197241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab690130-9208-49c1-95c1-18a2d4d14ef8 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:35.216103 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5747e91-4432-477d-a2a4-f57f4fc97ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:35.216748 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5747e91-4432-477d-a2a4-f57f4fc97ebb None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:35.217025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5747e91-4432-477d-a2a4-f57f4fc97ebb None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:35.217342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5747e91-4432-477d-a2a4-f57f4fc97ebb None admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:35.218600 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5747e91-4432-477d-a2a4-f57f4fc97ebb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:35.218900 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5747e91-4432-477d-a2a4-f57f4fc97ebb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:35.220378 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5747e91-4432-477d-a2a4-f57f4fc97ebb None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:35.237857 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 392/772] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:35:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:35.238698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5747e91-4432-477d-a2a4-f57f4fc97ebb None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:35.303349 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab690130-9208-49c1-95c1-18a2d4d14ef8 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:35.325860 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:35.337176 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8354349-daa9-45d4-81f3-cacb213d08a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:35.339469 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8354349-daa9-45d4-81f3-cacb213d08a6 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:35.339469 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8354349-daa9-45d4-81f3-cacb213d08a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:35.339469 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8354349-daa9-45d4-81f3-cacb213d08a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:35.339469 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8354349-daa9-45d4-81f3-cacb213d08a6 None admin] RBAC: Adding query filter params (interface=public, service_id=413bc371353b49ffb7707f6a0be50157, region_id=RegionOne) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:35.339469 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8354349-daa9-45d4-81f3-cacb213d08a6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:35.340842 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8354349-daa9-45d4-81f3-cacb213d08a6 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:35.340938 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5747e91-4432-477d-a2a4-f57f4fc97ebb None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:35.348799 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 381/773] 10.222.0.59 () {56 vars in 1251 bytes} [Fri Aug 26 00:35:35 2022] GET /identity/v3/endpoints?service_id=413bc371353b49ffb7707f6a0be50157&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:35.350023 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3151f-88e1-4a56-b600-715f0d48bfc8 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:35.350313 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3151f-88e1-4a56-b600-715f0d48bfc8 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:35.350591 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3151f-88e1-4a56-b600-715f0d48bfc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:35.368408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-f6e3151f-88e1-4a56-b600-715f0d48bfc8 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:35.369542 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-f6e3151f-88e1-4a56-b600-715f0d48bfc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:35.376090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964b384-a495-42f6-be1c-0a93f5ca25f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:35.376784 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964b384-a495-42f6-be1c-0a93f5ca25f4 None admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:35.377076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964b384-a495-42f6-be1c-0a93f5ca25f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:35.377336 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964b384-a495-42f6-be1c-0a93f5ca25f4 None admin] PATH_INFO: `/v3/projects/fde7ac3ed472433ab3bba56e5e091510/users/0602593263ca4b05a657d5b7697f932f/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:35.379223 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9964b384-a495-42f6-be1c-0a93f5ca25f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=fde7ac3ed472433ab3bba56e5e091510, user_id=0602593263ca4b05a657d5b7697f932f, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:35.380667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9964b384-a495-42f6-be1c-0a93f5ca25f4 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:35.408964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:35.431392 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 393/774] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Aug 26 00:35:35 2022] PUT /identity/v3/projects/fde7ac3ed472433ab3bba56e5e091510/users/0602593263ca4b05a657d5b7697f932f/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:35:35.432512 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964b384-a495-42f6-be1c-0a93f5ca25f4 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:35.441348 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 382/775] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:35:35.442027 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3151f-88e1-4a56-b600-715f0d48bfc8 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:35.561354 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3151f-88e1-4a56-b600-715f0d48bfc8 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:35.595020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964b384-a495-42f6-be1c-0a93f5ca25f4 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:35.602703 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bcb59-1187-4134-89dc-ba4346e8ae3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:35.603439 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bcb59-1187-4134-89dc-ba4346e8ae3e None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:35.603690 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bcb59-1187-4134-89dc-ba4346e8ae3e None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:35.603961 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bcb59-1187-4134-89dc-ba4346e8ae3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:35.616730 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52bcb59-1187-4134-89dc-ba4346e8ae3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:35.621861 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-b52bcb59-1187-4134-89dc-ba4346e8ae3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:35:35.621861 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:35.621861 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:35.621861 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:35.621861 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:35.621861 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:35.621861 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:35.621861 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:35.621861 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:35.621861 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:35.621861 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:35.621861 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:35.621861 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:35.621861 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:35.621861 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:35.621861 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:35.623031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:35.623031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:35.623031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:35.623031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:35.623031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:35.623031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:35.623031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:35.623031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:35.623031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:35.623031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:35.623031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:35.623031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:35.623031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:35.623031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:35.623031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:35.623031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:35.624702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:35.624702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:35.624702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:35.624702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:35.624702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:35.624702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:35.624702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:35.624702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:35.624702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:35.624702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:35.624702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:35:35.624702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:35.624702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 383/776] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:35:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:35.628610 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bcb59-1187-4134-89dc-ba4346e8ae3e None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:35.637872 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f534473-15f6-4c28-83e5-207f34e9a715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:35.638614 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f534473-15f6-4c28-83e5-207f34e9a715 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:35.639011 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f534473-15f6-4c28-83e5-207f34e9a715 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:35.639384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f534473-15f6-4c28-83e5-207f34e9a715 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:35.640870 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f534473-15f6-4c28-83e5-207f34e9a715 None admin] RBAC: Adding query filter params (role.id=bce78c7e0a534135a820de7f3588aa4d, scope.project.id=fde7ac3ed472433ab3bba56e5e091510, user.id=bbd9bbb083ab4f5e84e135ec5103cdae) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:35.641282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f534473-15f6-4c28-83e5-207f34e9a715 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:35.642981 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f534473-15f6-4c28-83e5-207f34e9a715 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:35.649002 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 394/777] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:35:35 2022] GET /identity/v3/role_assignments?user.id=bbd9bbb083ab4f5e84e135ec5103cdae&scope.project.id=fde7ac3ed472433ab3bba56e5e091510&role.id=bce78c7e0a534135a820de7f3588aa4d => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:35.650225 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f534473-15f6-4c28-83e5-207f34e9a715 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:35.664242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:35.669082 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:35.747858 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bcb59-1187-4134-89dc-ba4346e8ae3e None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:35.767078 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f534473-15f6-4c28-83e5-207f34e9a715 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:35.801223 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e3559-58de-41f7-a10e-b44f8f3fa319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:35.801964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e3559-58de-41f7-a10e-b44f8f3fa319 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:35.802311 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e3559-58de-41f7-a10e-b44f8f3fa319 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:35.802591 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e3559-58de-41f7-a10e-b44f8f3fa319 None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:35.803526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e3559-58de-41f7-a10e-b44f8f3fa319 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:35.805702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e3559-58de-41f7-a10e-b44f8f3fa319 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:35.808920 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe966a-a6ad-4abd-9e41-7547b262a33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:35.809639 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe966a-a6ad-4abd-9e41-7547b262a33f None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:35.809987 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe966a-a6ad-4abd-9e41-7547b262a33f None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:35.810328 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe966a-a6ad-4abd-9e41-7547b262a33f None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:35.811078 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe966a-a6ad-4abd-9e41-7547b262a33f None admin] RBAC: Adding query filter params (name=admin) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:35.812030 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe966a-a6ad-4abd-9e41-7547b262a33f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:35.812958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 384/778] 10.222.0.59 () {56 vars in 1087 bytes} [Fri Aug 26 00:35:35 2022] GET /identity/v3/services => generated 772 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:35.813636 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe966a-a6ad-4abd-9e41-7547b262a33f None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:35.814026 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e3559-58de-41f7-a10e-b44f8f3fa319 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:35.824329 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 395/779] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:35:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:35.830374 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe966a-a6ad-4abd-9e41-7547b262a33f None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:35.943860 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:35.992817 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:35.999830 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e3559-58de-41f7-a10e-b44f8f3fa319 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:36.010380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe966a-a6ad-4abd-9e41-7547b262a33f None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:36.048090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238c85d-943d-4828-9b5f-fc43cd7ecf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:36.048767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238c85d-943d-4828-9b5f-fc43cd7ecf7f None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:36.049050 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238c85d-943d-4828-9b5f-fc43cd7ecf7f None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:36.049335 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238c85d-943d-4828-9b5f-fc43cd7ecf7f None admin] PATH_INFO: `/v3/services/network` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:36.049919 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7238c85d-943d-4828-9b5f-fc43cd7ecf7f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:36.050992 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61399822-f494-40fb-b43b-abcca6cc1452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:36.051288 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7238c85d-943d-4828-9b5f-fc43cd7ecf7f None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:36.052253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61399822-f494-40fb-b43b-abcca6cc1452 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:36.052751 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61399822-f494-40fb-b43b-abcca6cc1452 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:36.053570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61399822-f494-40fb-b43b-abcca6cc1452 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:36.060946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-7238c85d-943d-4828-9b5f-fc43cd7ecf7f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 00:35:36.060946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:36.060946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 00:35:36.060946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 00:35:36.060946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 00:35:36.060946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 00:35:36.060946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 00:35:36.060946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:36.060946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 00:35:36.060946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:36.060946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:35:36.060946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:36.060946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:36.060946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:36.060946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:36.060946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:36.060946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:36.062006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:36.062006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:36.062006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:36.062006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:36.062006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:36.062006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:36.062006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 00:35:36.062006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 00:35:36.062006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 00:35:36.062006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 00:35:36.062006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:36.062006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:36.062006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:36.062006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:36.062006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:36.062006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:36.062892 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:36.062892 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:36.062892 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:36.062892 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:36.062892 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:36.062892 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:36.062892 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:36.062892 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:36.062892 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:36.062892 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:36.062892 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 00:35:36.062892 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:36.062892 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 00:35:36.062892 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:36.062892 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 385/780] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:35 2022] GET /identity/v3/services/network => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:36.070196 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238c85d-943d-4828-9b5f-fc43cd7ecf7f None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:36.074586 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61399822-f494-40fb-b43b-abcca6cc1452 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:36.077102 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61399822-f494-40fb-b43b-abcca6cc1452 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:36.094474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-61399822-f494-40fb-b43b-abcca6cc1452 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:36.094474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:36.094474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:36.094474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:36.094474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:36.094474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:36.094474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:36.094474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:36.094474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:36.094474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:36.094474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:36.094474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:36.094474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:36.094474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:36.094474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:36.094474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:36.095452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:36.095452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:36.095452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:36.095452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:36.095452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:36.095452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:36.095452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:36.095452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:36.095452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:36.095452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:36.095452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:36.095452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:36.095452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:36.095452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:36.095452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:36.095452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:36.095452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:36.096345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:36.096345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:36.096345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:36.098584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 396/781] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:36.104062 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61399822-f494-40fb-b43b-abcca6cc1452 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:36.236533 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61399822-f494-40fb-b43b-abcca6cc1452 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:36.262964 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:36.274807 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894f17a-fd6f-4146-a7c2-51285022d44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:36.275553 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894f17a-fd6f-4146-a7c2-51285022d44e None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:36.279257 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894f17a-fd6f-4146-a7c2-51285022d44e None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:36.279257 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894f17a-fd6f-4146-a7c2-51285022d44e None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:36.279257 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894f17a-fd6f-4146-a7c2-51285022d44e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:36.279257 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894f17a-fd6f-4146-a7c2-51285022d44e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:36.279257 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894f17a-fd6f-4146-a7c2-51285022d44e None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:36.293345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 397/782] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:36.299645 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894f17a-fd6f-4146-a7c2-51285022d44e None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:36.325899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238c85d-943d-4828-9b5f-fc43cd7ecf7f None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:36.382472 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc9313-3e2b-4839-b80a-0d56a26eb04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:36.383091 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc9313-3e2b-4839-b80a-0d56a26eb04b None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:36.383369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc9313-3e2b-4839-b80a-0d56a26eb04b None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:36.383639 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc9313-3e2b-4839-b80a-0d56a26eb04b None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:36.384383 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc9313-3e2b-4839-b80a-0d56a26eb04b None admin] RBAC: Adding query filter params (name=network) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:36.384717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc9313-3e2b-4839-b80a-0d56a26eb04b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:36.386071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc9313-3e2b-4839-b80a-0d56a26eb04b None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:36.398046 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 386/783] 10.222.0.59 () {56 vars in 1112 bytes} [Fri Aug 26 00:35:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:36.400289 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc9313-3e2b-4839-b80a-0d56a26eb04b None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:36.456665 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:36.483906 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894f17a-fd6f-4146-a7c2-51285022d44e None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:36.548824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5889fbac-7db0-40f6-90f1-3f1522d832b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:36.549466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5889fbac-7db0-40f6-90f1-3f1522d832b1 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:36.549746 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5889fbac-7db0-40f6-90f1-3f1522d832b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:36.550019 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5889fbac-7db0-40f6-90f1-3f1522d832b1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:36.563132 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5889fbac-7db0-40f6-90f1-3f1522d832b1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:36.564655 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5889fbac-7db0-40f6-90f1-3f1522d832b1 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:36.582398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-5889fbac-7db0-40f6-90f1-3f1522d832b1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 00:35:36.582398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:36.582398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:36.582398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:36.582398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:36.582398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:36.582398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:36.582398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:36.582398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:36.582398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:36.582398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:36.582398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:36.582398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:35:36.582398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:35:36.582398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:35:36.582398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:35:36.583335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:36.583335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:36.583335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:36.583335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:36.583335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:36.583335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:36.583335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:36.583335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:36.583335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:36.583335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:36.583335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:36.583335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:36.583335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:36.583335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:36.583335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:36.583335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:36.583335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:36.583335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:36.584297 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:36.584297 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:36.584297 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:35:36.584297 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:35:36.584297 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:35:36.584297 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:35:36.584297 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:35:36.584297 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:35:36.584297 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 00:35:36.584297 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:36.584892 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 398/784] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:35:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:36.589488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5889fbac-7db0-40f6-90f1-3f1522d832b1 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:36.592603 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc9313-3e2b-4839-b80a-0d56a26eb04b None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:36.652862 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:36.660323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279a05a-e348-4c72-9063-5994b9dac886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:36.661005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279a05a-e348-4c72-9063-5994b9dac886 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:36.661279 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279a05a-e348-4c72-9063-5994b9dac886 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:36.661546 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279a05a-e348-4c72-9063-5994b9dac886 None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:36.662225 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279a05a-e348-4c72-9063-5994b9dac886 None admin] RBAC: Adding query filter params (type=network) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:36.662525 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279a05a-e348-4c72-9063-5994b9dac886 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:36.666663 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279a05a-e348-4c72-9063-5994b9dac886 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:36.679430 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 387/785] 10.222.0.59 () {56 vars in 1112 bytes} [Fri Aug 26 00:35:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:36.685864 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279a05a-e348-4c72-9063-5994b9dac886 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:36.763395 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5889fbac-7db0-40f6-90f1-3f1522d832b1 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:36.834069 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185edbe4-8935-44b3-9cd8-570b68b8a392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:36.834613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185edbe4-8935-44b3-9cd8-570b68b8a392 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:36.834895 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185edbe4-8935-44b3-9cd8-570b68b8a392 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:36.835170 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185edbe4-8935-44b3-9cd8-570b68b8a392 None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:36.835910 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185edbe4-8935-44b3-9cd8-570b68b8a392 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:36.836213 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185edbe4-8935-44b3-9cd8-570b68b8a392 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:36.838239 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185edbe4-8935-44b3-9cd8-570b68b8a392 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:36.862725 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 399/786] 10.222.0.59 () {56 vars in 1135 bytes} [Fri Aug 26 00:35:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:35:36.864005 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185edbe4-8935-44b3-9cd8-570b68b8a392 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:36.877608 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:36.889568 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279a05a-e348-4c72-9063-5994b9dac886 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:36.938495 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da3fbe3-cfd8-45e1-b301-ead8ee02a1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:36.939107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3fbe3-cfd8-45e1-b301-ead8ee02a1e1 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:36.939380 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3fbe3-cfd8-45e1-b301-ead8ee02a1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:36.939645 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3fbe3-cfd8-45e1-b301-ead8ee02a1e1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:36.940340 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da3fbe3-cfd8-45e1-b301-ead8ee02a1e1 None admin] RBAC: Adding query filter params (interface=public, service_id=c9a1790abbe14018a8c7d101b58785d1, region_id=RegionOne) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:36.940666 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da3fbe3-cfd8-45e1-b301-ead8ee02a1e1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:36.942022 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da3fbe3-cfd8-45e1-b301-ead8ee02a1e1 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:36.953960 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 388/787] 10.222.0.59 () {56 vars in 1251 bytes} [Fri Aug 26 00:35:36 2022] GET /identity/v3/endpoints?service_id=c9a1790abbe14018a8c7d101b58785d1&interface=public®ion_id=RegionOne => generated 194 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:36.961757 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da3fbe3-cfd8-45e1-b301-ead8ee02a1e1 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:37.049202 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185edbe4-8935-44b3-9cd8-570b68b8a392 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:37.050905 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:37.105271 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8eff96-c58e-4cf1-a6c3-2fa35051cef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:37.105827 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8eff96-c58e-4cf1-a6c3-2fa35051cef1 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:37.106109 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8eff96-c58e-4cf1-a6c3-2fa35051cef1 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:37.106382 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8eff96-c58e-4cf1-a6c3-2fa35051cef1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:37.126936 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8eff96-c58e-4cf1-a6c3-2fa35051cef1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:37.128397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8eff96-c58e-4cf1-a6c3-2fa35051cef1 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:37.157726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-6f8eff96-c58e-4cf1-a6c3-2fa35051cef1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:37.157726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:37.157726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:37.157726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:37.157726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:37.157726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:37.157726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:37.157726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:37.157726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:37.157726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:37.157726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:37.157726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:37.157726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:37.157726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:37.157726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:37.157726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:37.158762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:37.158762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:37.158762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:37.158762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:37.158762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:37.158762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:37.158762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:37.158762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:37.158762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:37.158762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:37.158762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:37.158762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:37.158762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:37.158762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:37.158762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:37.158762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:37.158762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:37.159767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:37.159767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:37.159767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:37.160421 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 400/788] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:37.168607 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8eff96-c58e-4cf1-a6c3-2fa35051cef1 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:37.174243 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da3fbe3-cfd8-45e1-b301-ead8ee02a1e1 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:37.262819 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcecf11d-175f-4773-9a1d-1b298e452761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:37.263562 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcecf11d-175f-4773-9a1d-1b298e452761 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:37.263856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcecf11d-175f-4773-9a1d-1b298e452761 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:37.264135 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcecf11d-175f-4773-9a1d-1b298e452761 None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:37.265482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcecf11d-175f-4773-9a1d-1b298e452761 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:37.266863 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcecf11d-175f-4773-9a1d-1b298e452761 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:37.274958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 389/789] 10.222.0.59 () {56 vars in 1087 bytes} [Fri Aug 26 00:35:36 2022] GET /identity/v3/services => generated 772 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:37.285698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8eff96-c58e-4cf1-a6c3-2fa35051cef1 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:37.323225 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f5f81-976c-4019-9acf-0144e95ac3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:37.324230 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f5f81-976c-4019-9acf-0144e95ac3a5 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:37.324783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f5f81-976c-4019-9acf-0144e95ac3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:37.325275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f5f81-976c-4019-9acf-0144e95ac3a5 None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:37.326144 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f5f81-976c-4019-9acf-0144e95ac3a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:37.326730 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f5f81-976c-4019-9acf-0144e95ac3a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:37.329021 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f5f81-976c-4019-9acf-0144e95ac3a5 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:37.346763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 401/790] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:37.348496 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcecf11d-175f-4773-9a1d-1b298e452761 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:37.370431 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:37.460932 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcecf11d-175f-4773-9a1d-1b298e452761 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:37.498972 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e803b24-1923-4a8c-828b-858e15bbac87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:37.500162 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e803b24-1923-4a8c-828b-858e15bbac87 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:37.500442 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e803b24-1923-4a8c-828b-858e15bbac87 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:37.500751 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e803b24-1923-4a8c-828b-858e15bbac87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:37.507949 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e803b24-1923-4a8c-828b-858e15bbac87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:37.509520 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e803b24-1923-4a8c-828b-858e15bbac87 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:37.520974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-8e803b24-1923-4a8c-828b-858e15bbac87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:37.520974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:37.520974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:37.520974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:37.520974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:37.520974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:37.520974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:37.520974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:37.520974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:37.520974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:37.520974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:37.520974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:37.520974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:35:37.520974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:35:37.520974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:35:37.521879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:35:37.521879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:37.521879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:37.521879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:37.521879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:37.521879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:37.521879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:37.521879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:37.521879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:37.521879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:37.521879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:37.521879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:37.521879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:37.521879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:37.521879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:37.521879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:37.521879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:37.522775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:35:37.522775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:35:37.522775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:35:37.522775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:35:37.522775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:35:37.522775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:35:37.522775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:37.522775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:37.526865 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 390/791] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:37.528344 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f5f81-976c-4019-9acf-0144e95ac3a5 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:37.548863 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:37.635122 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b9621-b466-45f3-bf7a-c5f0cbfc1648 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:37.635721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b9621-b466-45f3-bf7a-c5f0cbfc1648 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:37.636191 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b9621-b466-45f3-bf7a-c5f0cbfc1648 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:37.638031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 391/792] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:37.641635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f5f81-976c-4019-9acf-0144e95ac3a5 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:37.644073 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697ddca-e15d-4d7b-97c1-b41ee667d214 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:37.644620 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697ddca-e15d-4d7b-97c1-b41ee667d214 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:37.645092 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697ddca-e15d-4d7b-97c1-b41ee667d214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:37.664321 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-a697ddca-e15d-4d7b-97c1-b41ee667d214 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:37.666108 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-a697ddca-e15d-4d7b-97c1-b41ee667d214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:37.691379 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c806fa-e7d3-4948-8768-1a80e39eaab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:37.692413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c806fa-e7d3-4948-8768-1a80e39eaab8 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:37.692915 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c806fa-e7d3-4948-8768-1a80e39eaab8 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:37.693405 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c806fa-e7d3-4948-8768-1a80e39eaab8 None admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:37.694296 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c806fa-e7d3-4948-8768-1a80e39eaab8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:37.695835 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c806fa-e7d3-4948-8768-1a80e39eaab8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:37.697550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c806fa-e7d3-4948-8768-1a80e39eaab8 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:37.731576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 402/793] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:35:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:37.738027 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c806fa-e7d3-4948-8768-1a80e39eaab8 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:37.792214 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 392/794] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 00:35:37.854558 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:37.894184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c806fa-e7d3-4948-8768-1a80e39eaab8 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:37.948770 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4660c1-0de0-4483-a70e-c62d2675e90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:37.949458 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4660c1-0de0-4483-a70e-c62d2675e90f None admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:37.949741 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4660c1-0de0-4483-a70e-c62d2675e90f None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:37.950021 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4660c1-0de0-4483-a70e-c62d2675e90f None admin] PATH_INFO: `/v3/projects/fde7ac3ed472433ab3bba56e5e091510/users/2f700502671446c797a566f07398a9fc/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:37.951040 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c3bf2-2c82-4b49-a1e8-c653fe0b858e None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:37.951621 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c3bf2-2c82-4b49-a1e8-c653fe0b858e None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:37.951967 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4660c1-0de0-4483-a70e-c62d2675e90f None admin] RBAC: Authorizing `identity:create_grant(project_id=fde7ac3ed472433ab3bba56e5e091510, user_id=2f700502671446c797a566f07398a9fc, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:37.952376 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c3bf2-2c82-4b49-a1e8-c653fe0b858e None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:37.953511 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4660c1-0de0-4483-a70e-c62d2675e90f None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:37.954418 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 393/795] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:37.959803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084e96f-4739-4a46-a224-91b0d00f5ca6 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:37.960339 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084e96f-4739-4a46-a224-91b0d00f5ca6 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:37.960831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084e96f-4739-4a46-a224-91b0d00f5ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:37.980323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-5084e96f-4739-4a46-a224-91b0d00f5ca6 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:37.981932 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-5084e96f-4739-4a46-a224-91b0d00f5ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:37.993267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:38.036209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 403/796] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Aug 26 00:35:37 2022] PUT /identity/v3/projects/fde7ac3ed472433ab3bba56e5e091510/users/2f700502671446c797a566f07398a9fc/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:35:38.063856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 394/797] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 00:35:38.136377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff12f95-837a-4d90-a9bd-f53cb70aba85 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:38.136722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff12f95-837a-4d90-a9bd-f53cb70aba85 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:38.136956 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff12f95-837a-4d90-a9bd-f53cb70aba85 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:38.138621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 404/798] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:38.143879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b0c5f-287c-44c9-b9fa-b361d6f0f7f4 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:38.144166 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b0c5f-287c-44c9-b9fa-b361d6f0f7f4 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:38.144520 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b0c5f-287c-44c9-b9fa-b361d6f0f7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:38.172875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-4e6b0c5f-287c-44c9-b9fa-b361d6f0f7f4 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:38.174075 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-4e6b0c5f-287c-44c9-b9fa-b361d6f0f7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:38.191060 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff12f95-837a-4d90-a9bd-f53cb70aba85 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:38.276662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 395/799] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:35:38.324279 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff12f95-837a-4d90-a9bd-f53cb70aba85 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:38.360271 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7727dc6-f215-443d-9010-a02dde3d3912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:38.360944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7727dc6-f215-443d-9010-a02dde3d3912 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:38.361225 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7727dc6-f215-443d-9010-a02dde3d3912 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:38.361503 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7727dc6-f215-443d-9010-a02dde3d3912 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:38.363375 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:38.372452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7727dc6-f215-443d-9010-a02dde3d3912 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:38.377842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-a7727dc6-f215-443d-9010-a02dde3d3912 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:35:38.377842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:38.377842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:38.377842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:38.377842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:38.377842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:38.377842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:38.377842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:38.377842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:38.377842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:38.377842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:38.377842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:38.377842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:38.377842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:38.377842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:38.377842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:38.381662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:38.381662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:38.381662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:38.381662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:38.381662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:38.381662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:38.381662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:38.381662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:38.381662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:38.381662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:38.381662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:38.381662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:38.381662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:38.381662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:38.381662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:38.381662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:38.382698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:38.382698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:38.382698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:38.382698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:38.382698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:38.382698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:38.382698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:38.382698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:38.382698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:38.382698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:38.382698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:35:38.382698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:38.382698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 405/800] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:35:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:38.385150 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b0c5f-287c-44c9-b9fa-b361d6f0f7f4 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:38.419919 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7727dc6-f215-443d-9010-a02dde3d3912 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:38.525699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:38.529181 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7727dc6-f215-443d-9010-a02dde3d3912 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:38.568590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69925257-96e2-4305-bf76-2c98e020ff88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:38.569242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69925257-96e2-4305-bf76-2c98e020ff88 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:38.569525 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69925257-96e2-4305-bf76-2c98e020ff88 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:38.569803 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69925257-96e2-4305-bf76-2c98e020ff88 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:38.590507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69925257-96e2-4305-bf76-2c98e020ff88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:38.598096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-69925257-96e2-4305-bf76-2c98e020ff88 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:38.598096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:38.598096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:38.598096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:38.598096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:38.598096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:38.598096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:38.598096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:38.598096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:38.598096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:38.598096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:38.598096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:38.598096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:38.598096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:38.598096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:38.598096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:38.599052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:38.599052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:38.599052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:38.599052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:38.599052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:38.599052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:38.599052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:38.599052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:38.599052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:38.599052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:38.599052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:38.599052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:38.599052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:38.599052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:38.599052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:38.599052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:38.599986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:38.599986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:38.599986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:38.599986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:38.599986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:38.599986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:38.599986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:38.599986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:38.599986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:38.599986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:38.599986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:38.599986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:38.599986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 406/801] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Aug 26 00:35:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:38.600705 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69925257-96e2-4305-bf76-2c98e020ff88 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:38.604220 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b0c5f-287c-44c9-b9fa-b361d6f0f7f4 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:38.652370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a386e3e-fedb-40ed-8a52-eaffbcbb07a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:38.653034 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a386e3e-fedb-40ed-8a52-eaffbcbb07a2 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:38.653318 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a386e3e-fedb-40ed-8a52-eaffbcbb07a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:38.653849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a386e3e-fedb-40ed-8a52-eaffbcbb07a2 None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:38.654581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a386e3e-fedb-40ed-8a52-eaffbcbb07a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:38.654885 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a386e3e-fedb-40ed-8a52-eaffbcbb07a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:38.656347 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a386e3e-fedb-40ed-8a52-eaffbcbb07a2 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:38.668376 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 396/802] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:35:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:35:38.669331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a386e3e-fedb-40ed-8a52-eaffbcbb07a2 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:38.681611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:38.739410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69925257-96e2-4305-bf76-2c98e020ff88 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:38.788800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198c6f72-f7d7-431a-bc45-08459ead09ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:38.789527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c6f72-f7d7-431a-bc45-08459ead09ce None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:38.789806 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c6f72-f7d7-431a-bc45-08459ead09ce None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:38.790076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c6f72-f7d7-431a-bc45-08459ead09ce None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:38.790801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198c6f72-f7d7-431a-bc45-08459ead09ce None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:38.792418 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198c6f72-f7d7-431a-bc45-08459ead09ce None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:38.804708 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a386e3e-fedb-40ed-8a52-eaffbcbb07a2 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:38.808149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-198c6f72-f7d7-431a-bc45-08459ead09ce None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 00:35:38.808149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:38.808149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 00:35:38.808149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 00:35:38.808149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 00:35:38.808149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 00:35:38.808149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 00:35:38.808149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:38.808149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 00:35:38.808149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:38.808149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:35:38.808149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:38.808149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:38.808149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:38.808149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:38.808149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:38.808149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:38.809342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:38.809342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:38.809342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:38.809342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:38.809342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:38.809342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:38.809342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 00:35:38.809342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 00:35:38.809342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 00:35:38.809342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 00:35:38.809342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:38.809342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:38.809342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:38.809342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:38.809342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:38.809342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:38.810398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:38.810398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:38.810398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:38.810398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:38.810398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:38.810398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:38.810398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:38.810398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:38.810398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:38.810398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:38.810398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 00:35:38.810398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:38.810398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 00:35:38.810398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:38.819495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 407/803] 10.222.0.59 () {56 vars in 1129 bytes} [Fri Aug 26 00:35:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:38.821162 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198c6f72-f7d7-431a-bc45-08459ead09ce None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:38.842009 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:38.849515 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911a51f-73d1-49bd-93e7-982f5f5727d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:38.850480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911a51f-73d1-49bd-93e7-982f5f5727d7 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:38.850827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911a51f-73d1-49bd-93e7-982f5f5727d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:38.851162 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911a51f-73d1-49bd-93e7-982f5f5727d7 None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:38.851890 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b911a51f-73d1-49bd-93e7-982f5f5727d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:38.852302 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b911a51f-73d1-49bd-93e7-982f5f5727d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:38.853950 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b911a51f-73d1-49bd-93e7-982f5f5727d7 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:38.868587 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 397/804] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Aug 26 00:35:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:38.869652 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911a51f-73d1-49bd-93e7-982f5f5727d7 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:38.951516 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198c6f72-f7d7-431a-bc45-08459ead09ce None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:38.970896 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:38.992897 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bbbe5d-f75c-4ead-b68c-d896be7a688a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:38.993863 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbbe5d-f75c-4ead-b68c-d896be7a688a None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:38.994352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbbe5d-f75c-4ead-b68c-d896be7a688a None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:38.994840 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbbe5d-f75c-4ead-b68c-d896be7a688a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:38.997990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911a51f-73d1-49bd-93e7-982f5f5727d7 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:39.006664 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bbbe5d-f75c-4ead-b68c-d896be7a688a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:39.008348 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bbbe5d-f75c-4ead-b68c-d896be7a688a None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:39.023690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-d5bbbe5d-f75c-4ead-b68c-d896be7a688a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:39.023690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:39.023690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:39.023690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:39.023690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:39.023690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:39.023690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:39.023690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:39.023690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:39.023690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:39.023690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:39.023690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:39.023690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:39.023690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:39.023690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:39.023690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:39.024951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:39.024951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:39.024951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:39.024951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:39.024951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:39.024951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:39.024951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:39.024951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:39.024951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:39.024951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:39.024951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:39.024951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:39.024951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:39.024951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:39.024951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:39.024951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:39.024951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:39.025850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:39.025850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:39.025850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:39.033052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 408/805] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:39.034108 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bbbe5d-f75c-4ead-b68c-d896be7a688a None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:39.039665 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14493496-b777-4ddc-9789-a7f1c8cf527a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:39.040284 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14493496-b777-4ddc-9789-a7f1c8cf527a None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:39.040590 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14493496-b777-4ddc-9789-a7f1c8cf527a None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:39.040865 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14493496-b777-4ddc-9789-a7f1c8cf527a None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:39.041489 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14493496-b777-4ddc-9789-a7f1c8cf527a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:39.041845 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14493496-b777-4ddc-9789-a7f1c8cf527a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:39.043267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14493496-b777-4ddc-9789-a7f1c8cf527a None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:39.053709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 398/806] 10.222.0.59 () {56 vars in 1124 bytes} [Fri Aug 26 00:35:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:39.053709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14493496-b777-4ddc-9789-a7f1c8cf527a None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:39.119043 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:39.167535 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14493496-b777-4ddc-9789-a7f1c8cf527a None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:39.174514 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bbbe5d-f75c-4ead-b68c-d896be7a688a None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:39.204593 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715db2c5-ae3d-420a-a9de-cc4d2efe49a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:39.205113 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715db2c5-ae3d-420a-a9de-cc4d2efe49a8 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:39.205389 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715db2c5-ae3d-420a-a9de-cc4d2efe49a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:39.205658 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715db2c5-ae3d-420a-a9de-cc4d2efe49a8 None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:39.206291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715db2c5-ae3d-420a-a9de-cc4d2efe49a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:39.206588 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715db2c5-ae3d-420a-a9de-cc4d2efe49a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:39.207942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715db2c5-ae3d-420a-a9de-cc4d2efe49a8 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:39.219771 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a7ab5-3293-413e-a87a-0dcfd6c5969d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:39.220414 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a7ab5-3293-413e-a87a-0dcfd6c5969d None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:39.220721 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a7ab5-3293-413e-a87a-0dcfd6c5969d None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:39.220991 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a7ab5-3293-413e-a87a-0dcfd6c5969d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:39.221291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 399/807] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:39.222144 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715db2c5-ae3d-420a-a9de-cc4d2efe49a8 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:39.235928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a7ab5-3293-413e-a87a-0dcfd6c5969d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:39.237661 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a7ab5-3293-413e-a87a-0dcfd6c5969d None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:39.258001 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-3c2a7ab5-3293-413e-a87a-0dcfd6c5969d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:39.258001 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:39.258001 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:39.258001 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:39.258001 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:39.258001 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:39.258001 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:39.258001 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:39.258001 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:39.258001 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:39.258001 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:39.258001 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:39.258001 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:39.258001 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:39.258001 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:39.258001 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:39.259031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:39.259031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:39.259031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:39.259031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:39.259031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:39.259031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:39.259031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:39.259031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:39.259031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:39.259031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:39.259031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:39.259031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:39.259031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:39.259031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:39.259031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:39.259031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:39.259031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:39.259946 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:39.259946 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:39.259946 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:39.265081 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 409/808] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:39.266050 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a7ab5-3293-413e-a87a-0dcfd6c5969d None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:39.302058 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:39.344668 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715db2c5-ae3d-420a-a9de-cc4d2efe49a8 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:39.368238 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:39.384862 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a7ab5-3293-413e-a87a-0dcfd6c5969d None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:39.391818 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e25cfd7-720c-47f0-926f-28bbbaf6671b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:39.392643 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25cfd7-720c-47f0-926f-28bbbaf6671b None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:39.393577 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25cfd7-720c-47f0-926f-28bbbaf6671b None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:39.394069 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25cfd7-720c-47f0-926f-28bbbaf6671b None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:39.394838 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e25cfd7-720c-47f0-926f-28bbbaf6671b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:39.395734 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e25cfd7-720c-47f0-926f-28bbbaf6671b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:39.397686 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e25cfd7-720c-47f0-926f-28bbbaf6671b None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:39.408570 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 400/809] 10.222.0.59 () {56 vars in 1124 bytes} [Fri Aug 26 00:35:39 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:39.410028 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e25cfd7-720c-47f0-926f-28bbbaf6671b None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:39.427844 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f625c3-5f2c-4090-b4db-e58e5dc074c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:39.428663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f625c3-5f2c-4090-b4db-e58e5dc074c8 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:39.429043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f625c3-5f2c-4090-b4db-e58e5dc074c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:39.429500 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f625c3-5f2c-4090-b4db-e58e5dc074c8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:39.442463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f625c3-5f2c-4090-b4db-e58e5dc074c8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:39.444201 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f625c3-5f2c-4090-b4db-e58e5dc074c8 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:39.456352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-64f625c3-5f2c-4090-b4db-e58e5dc074c8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 00:35:39.456352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:39.456352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:39.456352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:39.456352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:39.456352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:39.456352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:39.456352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:39.456352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:39.456352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:39.456352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:39.456352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:39.456352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:35:39.456352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:35:39.456352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:35:39.456352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:35:39.457411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:39.457411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:39.457411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:39.457411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:39.457411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:39.457411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:39.457411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:39.457411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:39.457411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:39.457411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:39.457411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:39.457411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:39.457411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:39.457411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:39.457411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:39.457411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:39.457411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:39.457411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:39.458373 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:39.458373 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:39.458373 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:35:39.458373 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:35:39.458373 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:35:39.458373 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:35:39.458373 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:35:39.458373 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:35:39.458373 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 00:35:39.458373 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:39.458373 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 410/810] 10.222.0.59 () {56 vars in 1093 bytes} [Fri Aug 26 00:35:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:39.458968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f625c3-5f2c-4090-b4db-e58e5dc074c8 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:39.544319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e25cfd7-720c-47f0-926f-28bbbaf6671b None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:39.544776 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:39.573921 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f625c3-5f2c-4090-b4db-e58e5dc074c8 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:39.583655 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb1181c-881e-4018-a0b6-8051b475739f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:39.584554 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb1181c-881e-4018-a0b6-8051b475739f None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:39.585033 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb1181c-881e-4018-a0b6-8051b475739f None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:39.585445 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb1181c-881e-4018-a0b6-8051b475739f None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:39.586362 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb1181c-881e-4018-a0b6-8051b475739f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:39.586823 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb1181c-881e-4018-a0b6-8051b475739f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:39.588436 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb1181c-881e-4018-a0b6-8051b475739f None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:39.602919 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 401/811] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:39.604041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb1181c-881e-4018-a0b6-8051b475739f None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:39.619762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ff846-5a90-44d9-a458-0bc25d8839fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:39.620781 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ff846-5a90-44d9-a458-0bc25d8839fe None admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:39.621300 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ff846-5a90-44d9-a458-0bc25d8839fe None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:39.621762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ff846-5a90-44d9-a458-0bc25d8839fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:39.622599 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ff846-5a90-44d9-a458-0bc25d8839fe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:39.624700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ff846-5a90-44d9-a458-0bc25d8839fe None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:39.652053 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:39.706857 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 411/812] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:35:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:35:39.707550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ff846-5a90-44d9-a458-0bc25d8839fe None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:39.715276 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb1181c-881e-4018-a0b6-8051b475739f None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:39.771879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898680d4-a898-4d4a-9f0b-8a86707eb6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:39.772804 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898680d4-a898-4d4a-9f0b-8a86707eb6e0 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:39.773265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898680d4-a898-4d4a-9f0b-8a86707eb6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:39.773701 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898680d4-a898-4d4a-9f0b-8a86707eb6e0 None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:39.774541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898680d4-a898-4d4a-9f0b-8a86707eb6e0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:39.774857 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898680d4-a898-4d4a-9f0b-8a86707eb6e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:39.776383 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898680d4-a898-4d4a-9f0b-8a86707eb6e0 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:39.799009 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 402/813] 10.222.0.59 () {56 vars in 1133 bytes} [Fri Aug 26 00:35:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:39.800239 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a75b0-67e3-4414-9ce3-f5a505dc7fad None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:39.800570 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a75b0-67e3-4414-9ce3-f5a505dc7fad None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:39.800851 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a75b0-67e3-4414-9ce3-f5a505dc7fad None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:39.806485 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 403/814] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:39.806485 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a75b0-67e3-4414-9ce3-f5a505dc7fad None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:39.869191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ff846-5a90-44d9-a458-0bc25d8839fe None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:39.915436 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:39.917830 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e58853-6d4c-43cc-a7f9-136e5eb1248b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:39.918493 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e58853-6d4c-43cc-a7f9-136e5eb1248b None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:39.918831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e58853-6d4c-43cc-a7f9-136e5eb1248b None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:39.919276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e58853-6d4c-43cc-a7f9-136e5eb1248b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:39.924364 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a75b0-67e3-4414-9ce3-f5a505dc7fad None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:39.928916 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e58853-6d4c-43cc-a7f9-136e5eb1248b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:39.931018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e58853-6d4c-43cc-a7f9-136e5eb1248b None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:39.935171 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:39.944324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-33e58853-6d4c-43cc-a7f9-136e5eb1248b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 00:35:39.944324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:39.944324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:39.944324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:39.944324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:39.944324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:39.944324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:39.944324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:39.944324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:39.944324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:39.944324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:39.944324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:39.944324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:35:39.944324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:35:39.944324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:35:39.944324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:35:39.945507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:39.945507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:39.945507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:39.945507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:39.945507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:39.945507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:39.945507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:39.945507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:39.945507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:39.945507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:39.945507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:39.945507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:39.945507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:39.945507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:39.945507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:39.945507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:39.945507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:39.945507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:39.946701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:39.946701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:39.946701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:35:39.946701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:35:39.946701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:35:39.946701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:35:39.946701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:35:39.946701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:35:39.946701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 00:35:39.946701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:39.948883 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 412/815] 10.222.0.59 () {56 vars in 1117 bytes} [Fri Aug 26 00:35:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:39.950633 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16a81f-0543-4ecb-9d95-788f2d641a87 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:39.951146 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16a81f-0543-4ecb-9d95-788f2d641a87 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:39.951429 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16a81f-0543-4ecb-9d95-788f2d641a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:39.968030 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482818e-a83f-40df-9372-9e7ed2a5699a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:39.969024 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482818e-a83f-40df-9372-9e7ed2a5699a None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:39.969553 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482818e-a83f-40df-9372-9e7ed2a5699a None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:39.969834 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482818e-a83f-40df-9372-9e7ed2a5699a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:39.971218 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-be16a81f-0543-4ecb-9d95-788f2d641a87 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:39.972536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-be16a81f-0543-4ecb-9d95-788f2d641a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:40.055243 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482818e-a83f-40df-9372-9e7ed2a5699a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:40.057148 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482818e-a83f-40df-9372-9e7ed2a5699a None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:40.066650 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 413/816] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:35:40.073523 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16a81f-0543-4ecb-9d95-788f2d641a87 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:40.165721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:40.174857 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 404/817] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:35:39 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 00:35:40.184603 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482818e-a83f-40df-9372-9e7ed2a5699a None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:40.289552 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16a81f-0543-4ecb-9d95-788f2d641a87 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:40.316389 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482818e-a83f-40df-9372-9e7ed2a5699a None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:40.325890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:40.334980 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff0662-1496-42d4-ba86-84079e8df95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:40.335625 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff0662-1496-42d4-ba86-84079e8df95e None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:40.335899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff0662-1496-42d4-ba86-84079e8df95e None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:40.336150 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff0662-1496-42d4-ba86-84079e8df95e None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:40.341118 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff0662-1496-42d4-ba86-84079e8df95e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:40.341118 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff0662-1496-42d4-ba86-84079e8df95e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:40.341118 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff0662-1496-42d4-ba86-84079e8df95e None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:40.359956 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a30ef-47f1-4214-a11b-160b2d0f925f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:40.360904 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 414/818] 10.222.0.59 () {56 vars in 1149 bytes} [Fri Aug 26 00:35:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:40.361650 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a30ef-47f1-4214-a11b-160b2d0f925f None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:40.362083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a30ef-47f1-4214-a11b-160b2d0f925f None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:40.362446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f52758-139a-4546-a711-124c1048a714 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:40.362673 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a30ef-47f1-4214-a11b-160b2d0f925f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:40.362837 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f52758-139a-4546-a711-124c1048a714 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:40.363191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f52758-139a-4546-a711-124c1048a714 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:40.368391 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 415/819] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:40.368391 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f52758-139a-4546-a711-124c1048a714 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:40.382914 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a30ef-47f1-4214-a11b-160b2d0f925f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:40.385396 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a30ef-47f1-4214-a11b-160b2d0f925f None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:40.404801 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-b20a30ef-47f1-4214-a11b-160b2d0f925f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:40.404801 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:40.404801 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:40.404801 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:40.404801 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:40.404801 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:40.404801 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:40.404801 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:40.404801 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:40.404801 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:40.404801 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:40.404801 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:40.404801 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:40.404801 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:40.404801 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:40.404801 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:40.406036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:40.406036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:40.406036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:40.406036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:40.406036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:40.406036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:40.406036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:40.406036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:40.406036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:40.406036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:40.406036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:40.406036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:40.406036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:40.406036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:40.406036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:40.406036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:40.406036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:40.407199 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:40.407199 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:40.407199 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:40.408886 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 405/820] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:40.410209 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1dfde-d3f4-4af5-8153-77c5f90c921a None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:40.410671 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1dfde-d3f4-4af5-8153-77c5f90c921a None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:40.410947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1dfde-d3f4-4af5-8153-77c5f90c921a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:40.431515 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-e8c1dfde-d3f4-4af5-8153-77c5f90c921a None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:40.432751 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-e8c1dfde-d3f4-4af5-8153-77c5f90c921a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:40.478498 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:40.488742 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f52758-139a-4546-a711-124c1048a714 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:40.515105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 406/821] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:35:40.515105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c1dfde-d3f4-4af5-8153-77c5f90c921a None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:40.533092 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba870054-7f02-434c-a19a-4125ee6bf081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:40.534115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba870054-7f02-434c-a19a-4125ee6bf081 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:40.534701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba870054-7f02-434c-a19a-4125ee6bf081 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:40.535050 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba870054-7f02-434c-a19a-4125ee6bf081 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:40.549764 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba870054-7f02-434c-a19a-4125ee6bf081 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:40.551232 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba870054-7f02-434c-a19a-4125ee6bf081 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:40.569780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-ba870054-7f02-434c-a19a-4125ee6bf081 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:40.569780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:40.569780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:40.569780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:40.569780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:40.569780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:40.569780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:40.569780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:40.569780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:40.569780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:40.569780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:40.569780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:40.569780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:40.569780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:40.569780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:40.569780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:40.571015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:40.571015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:40.571015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:40.571015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:40.571015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:40.571015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:40.571015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:40.571015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:40.571015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:40.571015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:40.571015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:40.571015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:40.571015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:40.571015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:40.571015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:40.571015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:40.571015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:40.571958 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:40.571958 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:40.571958 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:40.575965 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 416/822] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:40.576841 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba870054-7f02-434c-a19a-4125ee6bf081 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:40.618310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:40.632389 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c1dfde-d3f4-4af5-8153-77c5f90c921a None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:40.681063 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2afbd1-dbac-4c4a-85b7-d610405197ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:40.681980 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2afbd1-dbac-4c4a-85b7-d610405197ce None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:40.682267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2afbd1-dbac-4c4a-85b7-d610405197ce None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:40.682540 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2afbd1-dbac-4c4a-85b7-d610405197ce None admin] PATH_INFO: `/v3/services/network` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:40.683127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2afbd1-dbac-4c4a-85b7-d610405197ce None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:40.685222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2afbd1-dbac-4c4a-85b7-d610405197ce None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:40.688427 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba870054-7f02-434c-a19a-4125ee6bf081 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:40.697262 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-cd2afbd1-dbac-4c4a-85b7-d610405197ce None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 00:35:40.697262 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:40.697262 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 00:35:40.697262 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 00:35:40.697262 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 00:35:40.697262 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 00:35:40.697262 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 00:35:40.697262 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:40.697262 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 00:35:40.697262 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:40.697262 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:35:40.697262 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:40.697262 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:40.697262 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:40.697262 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:40.697262 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:40.697262 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:40.698341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:40.698341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:40.698341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:40.698341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:40.698341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:40.698341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:40.698341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 00:35:40.698341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 00:35:40.698341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 00:35:40.698341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 00:35:40.698341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:40.698341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:40.698341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:40.698341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:40.698341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:40.698341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:40.699777 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:40.699777 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:40.699777 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:40.699777 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:40.699777 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:40.699777 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:40.699777 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:40.699777 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:40.699777 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:40.699777 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:40.699777 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 00:35:40.699777 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:40.699777 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 00:35:40.699777 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:40.699777 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 407/823] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:40 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:40.700611 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2afbd1-dbac-4c4a-85b7-d610405197ce None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:40.743032 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25a52b-2162-42d3-a392-a597c87d62c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:40.744552 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25a52b-2162-42d3-a392-a597c87d62c8 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:40.745020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25a52b-2162-42d3-a392-a597c87d62c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:40.745990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:40.746956 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25a52b-2162-42d3-a392-a597c87d62c8 None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:40.748268 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db25a52b-2162-42d3-a392-a597c87d62c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:40.749666 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db25a52b-2162-42d3-a392-a597c87d62c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:40.751850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db25a52b-2162-42d3-a392-a597c87d62c8 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:40.772938 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 417/824] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:40.773771 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25a52b-2162-42d3-a392-a597c87d62c8 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:40.817928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2afbd1-dbac-4c4a-85b7-d610405197ce None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:40.854093 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196a256-c495-4f2d-a50d-ff624ec4dc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:40.854653 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196a256-c495-4f2d-a50d-ff624ec4dc4d None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:40.854937 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196a256-c495-4f2d-a50d-ff624ec4dc4d None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:40.855208 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196a256-c495-4f2d-a50d-ff624ec4dc4d None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:40.855962 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196a256-c495-4f2d-a50d-ff624ec4dc4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:40.856288 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196a256-c495-4f2d-a50d-ff624ec4dc4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:40.857784 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196a256-c495-4f2d-a50d-ff624ec4dc4d None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:40.876072 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 408/825] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:40.876072 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196a256-c495-4f2d-a50d-ff624ec4dc4d None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:40.890004 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:40.894261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25a52b-2162-42d3-a392-a597c87d62c8 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:40.936448 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72628bf-9975-43c3-ae7c-c4cbf2790ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:40.937111 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72628bf-9975-43c3-ae7c-c4cbf2790ccb None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:40.937858 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72628bf-9975-43c3-ae7c-c4cbf2790ccb None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:40.938143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72628bf-9975-43c3-ae7c-c4cbf2790ccb None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:40.938780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72628bf-9975-43c3-ae7c-c4cbf2790ccb None admin] RBAC: Adding query filter params (name=network) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:40.939084 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72628bf-9975-43c3-ae7c-c4cbf2790ccb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:40.940463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72628bf-9975-43c3-ae7c-c4cbf2790ccb None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:40.945342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 418/826] 10.222.0.59 () {56 vars in 1112 bytes} [Fri Aug 26 00:35:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:40.946031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72628bf-9975-43c3-ae7c-c4cbf2790ccb None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:40.982753 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:40.987530 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196a256-c495-4f2d-a50d-ff624ec4dc4d None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:41.028175 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a97d8a-cf40-420d-9bdf-bf81077f8f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:41.029085 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a97d8a-cf40-420d-9bdf-bf81077f8f09 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:41.029420 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a97d8a-cf40-420d-9bdf-bf81077f8f09 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:41.029690 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a97d8a-cf40-420d-9bdf-bf81077f8f09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:41.038681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a97d8a-cf40-420d-9bdf-bf81077f8f09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:41.040340 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a97d8a-cf40-420d-9bdf-bf81077f8f09 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:41.052256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-73a97d8a-cf40-420d-9bdf-bf81077f8f09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:41.052256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:41.052256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:41.052256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:41.052256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:41.052256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:41.052256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:41.052256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:41.052256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:41.052256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:41.052256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:41.052256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:41.052256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:35:41.052256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:35:41.052256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:35:41.053333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:35:41.053333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:41.053333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:41.053333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:41.053333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:41.053333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:41.053333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:41.053333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:41.053333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:41.053333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:41.053333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:41.053333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:41.053333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:41.053333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:41.053333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:41.053333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:41.053333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:41.054229 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:35:41.054229 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:35:41.054229 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:35:41.054229 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:35:41.054229 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:35:41.054229 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:35:41.054229 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:41.054229 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:41.054229 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 409/827] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:41.054738 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a97d8a-cf40-420d-9bdf-bf81077f8f09 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:41.058412 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72628bf-9975-43c3-ae7c-c4cbf2790ccb None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:41.094609 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a80ebc-ef96-4279-88d2-2d4ca215a455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:41.095306 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a80ebc-ef96-4279-88d2-2d4ca215a455 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:41.095661 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a80ebc-ef96-4279-88d2-2d4ca215a455 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:41.096031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a80ebc-ef96-4279-88d2-2d4ca215a455 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:41.106374 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a80ebc-ef96-4279-88d2-2d4ca215a455 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:41.111115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-61a80ebc-ef96-4279-88d2-2d4ca215a455 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:35:41.111115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:41.111115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:41.111115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:41.111115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:41.111115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:41.111115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:41.111115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:41.111115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:41.111115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:41.111115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:41.111115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:41.111115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:41.111115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:41.111115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:41.111115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:41.112051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:41.112051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:41.112051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:41.112051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:41.112051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:41.112051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:41.112051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:41.112051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:41.112051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:41.112051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:41.112051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:41.112051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:41.112051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:41.112051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:41.112051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:41.112051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:41.112952 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:41.112952 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:41.112952 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:41.112952 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:41.112952 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:41.112952 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:41.112952 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:41.112952 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:41.112952 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:41.112952 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:41.112952 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:35:41.112952 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:41.112952 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 419/828] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:35:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:41.114121 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a80ebc-ef96-4279-88d2-2d4ca215a455 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:41.143953 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:41.173784 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a97d8a-cf40-420d-9bdf-bf81077f8f09 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:41.209889 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75521564-900b-4b5f-9a57-4ef9a931f909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:41.210526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75521564-900b-4b5f-9a57-4ef9a931f909 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:41.210799 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75521564-900b-4b5f-9a57-4ef9a931f909 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:41.211070 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75521564-900b-4b5f-9a57-4ef9a931f909 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:41.217931 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75521564-900b-4b5f-9a57-4ef9a931f909 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:41.219359 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75521564-900b-4b5f-9a57-4ef9a931f909 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:41.222401 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a80ebc-ef96-4279-88d2-2d4ca215a455 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:41.230428 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-75521564-900b-4b5f-9a57-4ef9a931f909 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:41.230428 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:41.230428 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:41.230428 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:41.230428 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:41.230428 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:41.230428 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:41.230428 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:41.230428 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:41.230428 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:41.230428 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:41.230428 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:41.230428 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:35:41.230428 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:35:41.230428 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:35:41.231451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:35:41.231451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:41.231451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:41.231451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:41.231451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:41.231451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:41.231451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:41.231451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:41.231451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:41.231451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:41.231451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:41.231451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:41.231451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:41.231451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:41.231451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:41.231451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:41.231451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:41.232369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:35:41.232369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:35:41.232369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:35:41.232369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:35:41.232369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:35:41.232369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:35:41.232369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:41.232369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:41.236910 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 410/829] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:41.237464 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75521564-900b-4b5f-9a57-4ef9a931f909 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:41.260107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:41.262025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039714ea-582a-41a2-ad98-bf3029cd0a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:41.262650 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039714ea-582a-41a2-ad98-bf3029cd0a84 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:41.262935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039714ea-582a-41a2-ad98-bf3029cd0a84 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:41.263209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039714ea-582a-41a2-ad98-bf3029cd0a84 None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:41.263842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039714ea-582a-41a2-ad98-bf3029cd0a84 None admin] RBAC: Adding query filter params (type=network) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:41.264150 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039714ea-582a-41a2-ad98-bf3029cd0a84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:41.265703 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039714ea-582a-41a2-ad98-bf3029cd0a84 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:41.275079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 420/830] 10.222.0.59 () {56 vars in 1112 bytes} [Fri Aug 26 00:35:41 2022] GET /identity/v3/services?type=network => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:41.275958 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039714ea-582a-41a2-ad98-bf3029cd0a84 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:41.342877 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75521564-900b-4b5f-9a57-4ef9a931f909 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:41.368885 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:41.382091 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48761335-0ece-497d-927f-ee90998a3402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:41.388079 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48761335-0ece-497d-927f-ee90998a3402 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:41.388079 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48761335-0ece-497d-927f-ee90998a3402 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:41.388079 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48761335-0ece-497d-927f-ee90998a3402 None admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:41.388079 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48761335-0ece-497d-927f-ee90998a3402 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:41.388079 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48761335-0ece-497d-927f-ee90998a3402 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:41.388463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039714ea-582a-41a2-ad98-bf3029cd0a84 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:41.388587 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48761335-0ece-497d-927f-ee90998a3402 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:41.399319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 411/831] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:35:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:41.400132 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48761335-0ece-497d-927f-ee90998a3402 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:41.427594 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e445fc-47e1-479b-8b91-5b5ef9959a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:41.428460 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e445fc-47e1-479b-8b91-5b5ef9959a81 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:41.428774 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e445fc-47e1-479b-8b91-5b5ef9959a81 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:41.429200 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e445fc-47e1-479b-8b91-5b5ef9959a81 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:41.430186 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e445fc-47e1-479b-8b91-5b5ef9959a81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:41.430507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e445fc-47e1-479b-8b91-5b5ef9959a81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:41.433664 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e445fc-47e1-479b-8b91-5b5ef9959a81 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:41.444329 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 421/832] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:35:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:41.446019 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e445fc-47e1-479b-8b91-5b5ef9959a81 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:41.510371 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:41.532858 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48761335-0ece-497d-927f-ee90998a3402 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:41.569084 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e445fc-47e1-479b-8b91-5b5ef9959a81 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:41.574536 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68ae08-203b-4d42-aeae-f31535afe44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:41.575465 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68ae08-203b-4d42-aeae-f31535afe44b None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:41.575960 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68ae08-203b-4d42-aeae-f31535afe44b None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:41.577602 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68ae08-203b-4d42-aeae-f31535afe44b None admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:41.578545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef68ae08-203b-4d42-aeae-f31535afe44b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:41.579639 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef68ae08-203b-4d42-aeae-f31535afe44b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:41.582215 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef68ae08-203b-4d42-aeae-f31535afe44b None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:41.600929 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 412/833] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:35:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:41.602800 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68ae08-203b-4d42-aeae-f31535afe44b None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:41.608340 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27df244-31e3-468d-b8b8-407c6cb64312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:41.608966 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27df244-31e3-468d-b8b8-407c6cb64312 None admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:41.609247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27df244-31e3-468d-b8b8-407c6cb64312 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:41.609527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27df244-31e3-468d-b8b8-407c6cb64312 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:41.610903 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27df244-31e3-468d-b8b8-407c6cb64312 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:41.611569 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27df244-31e3-468d-b8b8-407c6cb64312 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:41.711827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68ae08-203b-4d42-aeae-f31535afe44b None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:41.716692 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 422/834] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:35:41 2022] POST /identity/v3/endpoints => generated 337 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:35:41.717736 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27df244-31e3-468d-b8b8-407c6cb64312 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:41.723735 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:41.780756 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26b23b-41c1-49ed-abc0-12d1aa21bd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:41.782347 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26b23b-41c1-49ed-abc0-12d1aa21bd56 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:41.782710 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26b23b-41c1-49ed-abc0-12d1aa21bd56 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:41.783010 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26b23b-41c1-49ed-abc0-12d1aa21bd56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:41.783751 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e26b23b-41c1-49ed-abc0-12d1aa21bd56 None admin] RBAC: Adding query filter params (role.id=1a33e3c8cb8347feb9498595886cc621, scope.project.id=fde7ac3ed472433ab3bba56e5e091510, user.id=0602593263ca4b05a657d5b7697f932f) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:41.784111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e26b23b-41c1-49ed-abc0-12d1aa21bd56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:41.785674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e26b23b-41c1-49ed-abc0-12d1aa21bd56 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:41.789708 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:41.806142 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 413/835] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:35:41 2022] GET /identity/v3/role_assignments?user.id=0602593263ca4b05a657d5b7697f932f&scope.project.id=fde7ac3ed472433ab3bba56e5e091510&role.id=1a33e3c8cb8347feb9498595886cc621 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:41.807390 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26b23b-41c1-49ed-abc0-12d1aa21bd56 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:41.915753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27df244-31e3-468d-b8b8-407c6cb64312 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:41.950389 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26b23b-41c1-49ed-abc0-12d1aa21bd56 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:41.960905 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef51ae-7e5e-4f37-8059-53cc0496471d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:41.961555 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef51ae-7e5e-4f37-8059-53cc0496471d None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:41.961886 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef51ae-7e5e-4f37-8059-53cc0496471d None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:41.962476 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef51ae-7e5e-4f37-8059-53cc0496471d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:41.980179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef51ae-7e5e-4f37-8059-53cc0496471d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:41.982605 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef51ae-7e5e-4f37-8059-53cc0496471d None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:41.997578 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a644b2f-06e4-4962-a9ef-5d5947042409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:41.997772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-33ef51ae-7e5e-4f37-8059-53cc0496471d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:41.997772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:41.997772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:41.997772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:41.997772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:41.997772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:41.997772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:41.997772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:41.997772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:41.997772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:41.997772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:41.997772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:41.997772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:41.997772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:41.997772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:41.997772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:41.998676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:41.998676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:41.998676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:41.998676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:41.998676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:41.998676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:41.998676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:41.998676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:41.998676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:41.998676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:41.998676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:41.998676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:41.998676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:41.998676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:41.998676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:41.998676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:41.998676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:42.000163 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:42.000163 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:42.000163 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:42.000335 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a644b2f-06e4-4962-a9ef-5d5947042409 None admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:42.000335 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a644b2f-06e4-4962-a9ef-5d5947042409 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:42.000335 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a644b2f-06e4-4962-a9ef-5d5947042409 None admin] PATH_INFO: `/v3/projects/fde7ac3ed472433ab3bba56e5e091510/users/bbd9bbb083ab4f5e84e135ec5103cdae/roles/bce78c7e0a534135a820de7f3588aa4d` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:42.000569 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 423/836] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:42.003077 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1bc6bc-ce4a-44b3-a641-c48fe88c2901 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:42.003077 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1bc6bc-ce4a-44b3-a641-c48fe88c2901 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:42.003077 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1bc6bc-ce4a-44b3-a641-c48fe88c2901 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:42.003077 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 424/837] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:42.006313 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1bc6bc-ce4a-44b3-a641-c48fe88c2901 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:42.018512 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a644b2f-06e4-4962-a9ef-5d5947042409 None admin] RBAC: Authorizing `identity:create_grant(project_id=fde7ac3ed472433ab3bba56e5e091510, user_id=bbd9bbb083ab4f5e84e135ec5103cdae, role_id=bce78c7e0a534135a820de7f3588aa4d)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:42.019951 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a644b2f-06e4-4962-a9ef-5d5947042409 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:42.068148 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:42.113700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1bc6bc-ce4a-44b3-a641-c48fe88c2901 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:42.117558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 414/838] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Aug 26 00:35:41 2022] PUT /identity/v3/projects/fde7ac3ed472433ab3bba56e5e091510/users/bbd9bbb083ab4f5e84e135ec5103cdae/roles/bce78c7e0a534135a820de7f3588aa4d => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:35:42.129988 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942df87f-6feb-4d52-b6fd-c057ade28934 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:42.130270 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942df87f-6feb-4d52-b6fd-c057ade28934 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:42.130550 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942df87f-6feb-4d52-b6fd-c057ade28934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:42.157956 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-942df87f-6feb-4d52-b6fd-c057ade28934 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:42.159325 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-942df87f-6feb-4d52-b6fd-c057ade28934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:42.163617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5e8e7-968e-49f3-8127-58ca91ef08d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:42.164688 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5e8e7-968e-49f3-8127-58ca91ef08d6 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:42.165151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5e8e7-968e-49f3-8127-58ca91ef08d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:42.165578 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5e8e7-968e-49f3-8127-58ca91ef08d6 None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:42.166461 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5e8e7-968e-49f3-8127-58ca91ef08d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:42.166920 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5e8e7-968e-49f3-8127-58ca91ef08d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:42.168512 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5e8e7-968e-49f3-8127-58ca91ef08d6 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:42.188698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 425/839] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:35:42.193973 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5e8e7-968e-49f3-8127-58ca91ef08d6 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:42.226878 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:42.257187 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 415/840] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:35:42.321780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5e8e7-968e-49f3-8127-58ca91ef08d6 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:42.341582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:42.361142 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86fcc26-89a3-46e9-94de-f2b1a5655ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:42.361783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fcc26-89a3-46e9-94de-f2b1a5655ed5 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:42.362068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fcc26-89a3-46e9-94de-f2b1a5655ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:42.362346 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fcc26-89a3-46e9-94de-f2b1a5655ed5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:42.370007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86fcc26-89a3-46e9-94de-f2b1a5655ed5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:42.371460 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86fcc26-89a3-46e9-94de-f2b1a5655ed5 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:42.384446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-d86fcc26-89a3-46e9-94de-f2b1a5655ed5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 00:35:42.384446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:42.384446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:42.384446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:42.384446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:42.384446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:42.384446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:42.384446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:42.384446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:42.384446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:42.384446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:42.384446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:42.384446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:35:42.384446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:35:42.384446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:35:42.384446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:35:42.385483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:42.385483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:42.385483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:42.385483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:42.385483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:42.385483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:42.385483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:42.385483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:42.385483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:42.385483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:42.385483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:42.385483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:42.385483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:42.385483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:42.385483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:42.385483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:42.385483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:42.385483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:42.386469 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:42.386469 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:42.386469 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:35:42.386469 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:35:42.386469 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:35:42.386469 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:35:42.386469 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:35:42.386469 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:35:42.386469 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 00:35:42.386469 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:42.386469 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 426/841] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:35:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:42.394570 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942df87f-6feb-4d52-b6fd-c057ade28934 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:42.506349 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942df87f-6feb-4d52-b6fd-c057ade28934 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:42.542449 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d404507f-52e2-4132-86c2-1f6d47afd490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:42.543064 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404507f-52e2-4132-86c2-1f6d47afd490 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:42.543345 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404507f-52e2-4132-86c2-1f6d47afd490 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:42.543616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404507f-52e2-4132-86c2-1f6d47afd490 None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:42.544406 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d404507f-52e2-4132-86c2-1f6d47afd490 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:42.544743 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d404507f-52e2-4132-86c2-1f6d47afd490 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:42.546794 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d404507f-52e2-4132-86c2-1f6d47afd490 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:42.548576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86fcc26-89a3-46e9-94de-f2b1a5655ed5 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:42.565756 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 416/842] 10.222.0.59 () {56 vars in 1135 bytes} [Fri Aug 26 00:35:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:42.571207 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d404507f-52e2-4132-86c2-1f6d47afd490 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:42.615286 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:42.664233 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86fcc26-89a3-46e9-94de-f2b1a5655ed5 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:42.694846 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d404507f-52e2-4132-86c2-1f6d47afd490 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:42.700032 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb2684-4949-4541-bd73-69b9ee90481f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:42.700825 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb2684-4949-4541-bd73-69b9ee90481f None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:42.701178 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb2684-4949-4541-bd73-69b9ee90481f None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:42.701521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb2684-4949-4541-bd73-69b9ee90481f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:42.702242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb2684-4949-4541-bd73-69b9ee90481f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:42.703740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb2684-4949-4541-bd73-69b9ee90481f None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:42.712983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-2aeb2684-4949-4541-bd73-69b9ee90481f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 00:35:42.712983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:42.712983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 00:35:42.712983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 00:35:42.712983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 00:35:42.712983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 00:35:42.712983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 00:35:42.712983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:42.712983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 00:35:42.712983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:42.712983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:35:42.712983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:42.712983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:42.712983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:42.712983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:42.712983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:42.712983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:42.714003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:42.714003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:42.714003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:42.714003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:42.714003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:42.714003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:42.714003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 00:35:42.714003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 00:35:42.714003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 00:35:42.714003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 00:35:42.714003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:42.714003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:42.714003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:42.714003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:42.714003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:42.714003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:42.714887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:42.714887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:42.714887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:42.714887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:42.714887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:42.714887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:42.714887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:42.714887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:42.714887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:42.714887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:42.714887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 00:35:42.714887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:42.714887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 00:35:42.714887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:42.714887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 427/843] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Aug 26 00:35:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:42.722619 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb2684-4949-4541-bd73-69b9ee90481f None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:42.734081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb801d02-6452-4524-a358-a8e84b66e9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:42.734695 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb801d02-6452-4524-a358-a8e84b66e9fa None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:42.734969 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb801d02-6452-4524-a358-a8e84b66e9fa None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:42.735235 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb801d02-6452-4524-a358-a8e84b66e9fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:42.781124 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:42.809245 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb801d02-6452-4524-a358-a8e84b66e9fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:42.810688 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb801d02-6452-4524-a358-a8e84b66e9fa None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:42.853994 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:42.864755 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb2684-4949-4541-bd73-69b9ee90481f None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:42.903163 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a85e96-f926-4983-b140-4668fb0fec5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:42.904035 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a85e96-f926-4983-b140-4668fb0fec5f None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:42.904317 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a85e96-f926-4983-b140-4668fb0fec5f None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:42.904669 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a85e96-f926-4983-b140-4668fb0fec5f None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:42.905992 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a85e96-f926-4983-b140-4668fb0fec5f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:42.906297 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a85e96-f926-4983-b140-4668fb0fec5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:42.907998 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a85e96-f926-4983-b140-4668fb0fec5f None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:42.917534 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 428/844] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Aug 26 00:35:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:35:42.918216 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a85e96-f926-4983-b140-4668fb0fec5f None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:42.924597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 417/845] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:35:42 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 00:35:42.931707 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb801d02-6452-4524-a358-a8e84b66e9fa None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:43.036865 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb801d02-6452-4524-a358-a8e84b66e9fa None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:43.054140 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a85e96-f926-4983-b140-4668fb0fec5f None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:43.057506 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:43.074493 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45cfa5-5a93-4782-a0d2-d9e1f449f0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:43.075470 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45cfa5-5a93-4782-a0d2-d9e1f449f0db None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:43.075864 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45cfa5-5a93-4782-a0d2-d9e1f449f0db None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:43.076291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45cfa5-5a93-4782-a0d2-d9e1f449f0db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:43.084506 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:43.088348 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45cfa5-5a93-4782-a0d2-d9e1f449f0db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:43.089972 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45cfa5-5a93-4782-a0d2-d9e1f449f0db None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:43.102804 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb77f9e-85c7-4ae2-a55c-c9d5d510683b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:43.103627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb77f9e-85c7-4ae2-a55c-c9d5d510683b None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:43.104058 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb77f9e-85c7-4ae2-a55c-c9d5d510683b None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:43.104503 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb77f9e-85c7-4ae2-a55c-c9d5d510683b None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:43.105283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb77f9e-85c7-4ae2-a55c-c9d5d510683b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:43.105755 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb77f9e-85c7-4ae2-a55c-c9d5d510683b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:43.106103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-8a45cfa5-5a93-4782-a0d2-d9e1f449f0db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:43.106103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:43.106103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:43.106103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:43.106103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:43.106103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:43.106103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:43.106103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:43.106103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:43.106103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:43.106103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:43.106103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:43.106103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:43.106103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:43.106103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:43.106103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:43.107232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:43.107232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:43.107232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:43.107232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:43.107232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:43.107232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:43.107232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:43.107232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:43.107232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:43.107232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:43.107232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:43.107232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:43.107232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:43.107232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:43.107232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:43.107232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:43.107232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:43.108407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:43.108407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:43.108407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:43.109781 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 418/846] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:43.110128 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb77f9e-85c7-4ae2-a55c-c9d5d510683b None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:43.114891 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45cfa5-5a93-4782-a0d2-d9e1f449f0db None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:43.120873 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 429/847] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Aug 26 00:35:42 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:43.234930 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45cfa5-5a93-4782-a0d2-d9e1f449f0db None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:43.274605 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90320bae-7bf4-46d7-b8d9-2a5c80304f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:43.275652 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90320bae-7bf4-46d7-b8d9-2a5c80304f9a None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:43.276147 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90320bae-7bf4-46d7-b8d9-2a5c80304f9a None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:43.276657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90320bae-7bf4-46d7-b8d9-2a5c80304f9a None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:43.277578 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90320bae-7bf4-46d7-b8d9-2a5c80304f9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:43.277885 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90320bae-7bf4-46d7-b8d9-2a5c80304f9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:43.279536 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90320bae-7bf4-46d7-b8d9-2a5c80304f9a None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:43.292809 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 419/848] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:43.298601 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb77f9e-85c7-4ae2-a55c-c9d5d510683b None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:43.412409 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb77f9e-85c7-4ae2-a55c-c9d5d510683b None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:43.455622 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffea555-64ff-47ad-96e9-7115c34d4474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:43.456273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffea555-64ff-47ad-96e9-7115c34d4474 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:43.456579 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffea555-64ff-47ad-96e9-7115c34d4474 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:43.456855 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffea555-64ff-47ad-96e9-7115c34d4474 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:43.464046 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffea555-64ff-47ad-96e9-7115c34d4474 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:43.465529 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffea555-64ff-47ad-96e9-7115c34d4474 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:43.477089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-fffea555-64ff-47ad-96e9-7115c34d4474 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:43.477089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:43.477089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:43.477089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:43.477089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:43.477089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:43.477089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:43.477089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:43.477089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:43.477089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:43.477089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:43.477089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:43.477089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:35:43.477089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:35:43.477089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:35:43.478003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:35:43.478003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:43.478003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:43.478003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:43.478003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:43.478003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:43.478003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:43.478003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:43.478003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:43.478003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:43.478003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:43.478003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:43.478003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:43.478003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:43.478003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:43.478003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:43.478003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:43.479115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:35:43.479115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:35:43.479115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:35:43.479115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:35:43.479115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:35:43.479115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:35:43.479115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:43.479115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:43.479115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 430/849] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:43.486628 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90320bae-7bf4-46d7-b8d9-2a5c80304f9a None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:43.548556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:43.603618 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90320bae-7bf4-46d7-b8d9-2a5c80304f9a None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:43.638461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14193fd6-b7df-4ab4-8162-323931c6eed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:43.639774 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14193fd6-b7df-4ab4-8162-323931c6eed9 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:43.640261 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14193fd6-b7df-4ab4-8162-323931c6eed9 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:43.640802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14193fd6-b7df-4ab4-8162-323931c6eed9 None admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:43.641695 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14193fd6-b7df-4ab4-8162-323931c6eed9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:43.642213 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14193fd6-b7df-4ab4-8162-323931c6eed9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:43.643857 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14193fd6-b7df-4ab4-8162-323931c6eed9 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:43.657135 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 420/850] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:35:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:35:43.663345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffea555-64ff-47ad-96e9-7115c34d4474 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:43.694443 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:43.771205 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffea555-64ff-47ad-96e9-7115c34d4474 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:43.808159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa492f1-abfa-4264-a8ef-7e8e33582781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:43.809033 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa492f1-abfa-4264-a8ef-7e8e33582781 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:43.809465 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa492f1-abfa-4264-a8ef-7e8e33582781 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:43.809891 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa492f1-abfa-4264-a8ef-7e8e33582781 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:43.810772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa492f1-abfa-4264-a8ef-7e8e33582781 None admin] RBAC: Adding query filter params (role.id=1a33e3c8cb8347feb9498595886cc621, scope.project.id=fde7ac3ed472433ab3bba56e5e091510, user.id=2f700502671446c797a566f07398a9fc) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:43.811287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa492f1-abfa-4264-a8ef-7e8e33582781 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:43.813584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa492f1-abfa-4264-a8ef-7e8e33582781 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:43.822584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 431/851] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:35:43 2022] GET /identity/v3/role_assignments?user.id=2f700502671446c797a566f07398a9fc&scope.project.id=fde7ac3ed472433ab3bba56e5e091510&role.id=1a33e3c8cb8347feb9498595886cc621 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:43.916742 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d01f5-23d6-4462-80f2-185610b8f30c None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:43.917195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d01f5-23d6-4462-80f2-185610b8f30c None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:43.917589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d01f5-23d6-4462-80f2-185610b8f30c None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:43.919305 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 421/852] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:43.926834 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb19f4-30dc-4e91-9eb7-7ec36afd32f2 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:43.927256 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb19f4-30dc-4e91-9eb7-7ec36afd32f2 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:43.927632 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb19f4-30dc-4e91-9eb7-7ec36afd32f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:43.946728 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-f5eb19f4-30dc-4e91-9eb7-7ec36afd32f2 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:43.948007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f5eb19f4-30dc-4e91-9eb7-7ec36afd32f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:44.018018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 432/853] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:35:44.138793 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9fad1-99b4-4cc3-866e-928f757081e3 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:44.139136 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9fad1-99b4-4cc3-866e-928f757081e3 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:44.139372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9fad1-99b4-4cc3-866e-928f757081e3 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:44.141810 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 422/854] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:44.147071 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60062e-0b90-494a-a8b4-0ccd23a2093b None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:44.147367 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60062e-0b90-494a-a8b4-0ccd23a2093b None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:44.147645 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60062e-0b90-494a-a8b4-0ccd23a2093b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:44.166502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-8d60062e-0b90-494a-a8b4-0ccd23a2093b None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:44.167706 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-8d60062e-0b90-494a-a8b4-0ccd23a2093b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:44.186968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:44.241073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 433/855] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:35:44.287468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9fad1-99b4-4cc3-866e-928f757081e3 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:44.391482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9fad1-99b4-4cc3-866e-928f757081e3 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:44.408841 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:44.431461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f0ac8-c5a5-40e8-8fc9-bfeb375ca9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:44.432107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f0ac8-c5a5-40e8-8fc9-bfeb375ca9d0 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:44.432392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f0ac8-c5a5-40e8-8fc9-bfeb375ca9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:44.432694 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f0ac8-c5a5-40e8-8fc9-bfeb375ca9d0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:44.433287 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f0ac8-c5a5-40e8-8fc9-bfeb375ca9d0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:44.434648 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f0ac8-c5a5-40e8-8fc9-bfeb375ca9d0 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:44.444633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-fb1f0ac8-c5a5-40e8-8fc9-bfeb375ca9d0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 00:35:44.444633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:44.444633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 00:35:44.444633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 00:35:44.444633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 00:35:44.444633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 00:35:44.444633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 00:35:44.444633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:44.444633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 00:35:44.444633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:44.444633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:35:44.444633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:44.444633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:44.444633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:44.444633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:44.444633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:44.445597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:44.445597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:44.445597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:44.445597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:44.445597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:44.445597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:44.445597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:44.445597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 00:35:44.445597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 00:35:44.445597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 00:35:44.445597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 00:35:44.445597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:44.445597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:44.445597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:44.445597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:44.445597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:44.445597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:44.446503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:44.446503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:44.446503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:44.446503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:44.446503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:44.446503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:44.446503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:44.446503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:44.446503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:44.446503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:44.446503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 00:35:44.446503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:44.446503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 00:35:44.446503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:44.449776 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 423/856] 10.222.0.59 () {56 vars in 1132 bytes} [Fri Aug 26 00:35:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:44.453561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60062e-0b90-494a-a8b4-0ccd23a2093b None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:44.539151 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f0ac8-c5a5-40e8-8fc9-bfeb375ca9d0 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:44.558161 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60062e-0b90-494a-a8b4-0ccd23a2093b None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:44.597864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee5a27-5bf0-4617-b43d-e48ae19524bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:44.598532 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee5a27-5bf0-4617-b43d-e48ae19524bc None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:44.598813 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee5a27-5bf0-4617-b43d-e48ae19524bc None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:44.599084 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee5a27-5bf0-4617-b43d-e48ae19524bc None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:44.599763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee5a27-5bf0-4617-b43d-e48ae19524bc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:44.600091 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee5a27-5bf0-4617-b43d-e48ae19524bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:44.601625 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee5a27-5bf0-4617-b43d-e48ae19524bc None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:44.606740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 434/857] 10.222.0.59 () {56 vars in 1126 bytes} [Fri Aug 26 00:35:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:44.612427 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee5a27-5bf0-4617-b43d-e48ae19524bc None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:44.631776 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:44.672594 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f0ac8-c5a5-40e8-8fc9-bfeb375ca9d0 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:44.721258 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8218d82-e1ea-4561-9c7a-bbd32666414b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:44.722038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8218d82-e1ea-4561-9c7a-bbd32666414b None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:44.722432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8218d82-e1ea-4561-9c7a-bbd32666414b None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:44.722766 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8218d82-e1ea-4561-9c7a-bbd32666414b None admin] PATH_INFO: `/v3/roles/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:44.730458 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee5a27-5bf0-4617-b43d-e48ae19524bc None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:44.735473 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8218d82-e1ea-4561-9c7a-bbd32666414b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:44.741059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-e8218d82-e1ea-4561-9c7a-bbd32666414b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:44.741059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:44.741059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:44.741059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:44.741059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:44.741059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:44.741059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:44.741059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:44.741059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:44.741059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:44.741059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:44.741059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:44.741059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:44.741059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:44.741059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:44.741059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:44.742036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:44.742036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:44.742036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:44.742036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:44.742036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:44.742036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:44.742036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:44.742036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:44.742036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:44.742036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:44.742036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:44.742036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:44.742036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:44.742036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:44.742036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:44.742036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:44.748301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:44.748301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:44.748301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:44.748301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:44.748301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:44.748301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:44.748301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:44.748301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:44.748301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:44.748301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:44.748301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:35:44.748301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:44.748301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 424/858] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Aug 26 00:35:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:44.750458 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8218d82-e1ea-4561-9c7a-bbd32666414b None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:44.780993 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b77d08-dab8-4e80-9357-6d0d9e1b8d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:44.781662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b77d08-dab8-4e80-9357-6d0d9e1b8d53 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:44.781944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b77d08-dab8-4e80-9357-6d0d9e1b8d53 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:44.782217 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b77d08-dab8-4e80-9357-6d0d9e1b8d53 None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:44.783124 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b77d08-dab8-4e80-9357-6d0d9e1b8d53 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:44.783483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b77d08-dab8-4e80-9357-6d0d9e1b8d53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:44.785673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b77d08-dab8-4e80-9357-6d0d9e1b8d53 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:44.799349 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 435/859] 10.222.0.59 () {56 vars in 1126 bytes} [Fri Aug 26 00:35:44 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:44.799554 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:44.871802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8218d82-e1ea-4561-9c7a-bbd32666414b None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:44.911064 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa5d05-44a3-447d-80d5-8078ab1a2790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:44.911695 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa5d05-44a3-447d-80d5-8078ab1a2790 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:44.911976 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa5d05-44a3-447d-80d5-8078ab1a2790 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:44.912248 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa5d05-44a3-447d-80d5-8078ab1a2790 None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:44.912964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa5d05-44a3-447d-80d5-8078ab1a2790 None admin] RBAC: Adding query filter params (name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:44.913269 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa5d05-44a3-447d-80d5-8078ab1a2790 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:44.914747 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa5d05-44a3-447d-80d5-8078ab1a2790 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:44.928782 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 425/860] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Aug 26 00:35:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:35:44.930426 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b77d08-dab8-4e80-9357-6d0d9e1b8d53 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:45.020748 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:45.041457 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b77d08-dab8-4e80-9357-6d0d9e1b8d53 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:45.078195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d6343-6a7e-4031-82d2-553080786437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:45.078836 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d6343-6a7e-4031-82d2-553080786437 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:45.079117 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d6343-6a7e-4031-82d2-553080786437 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:45.079390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d6343-6a7e-4031-82d2-553080786437 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:45.089930 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d6343-6a7e-4031-82d2-553080786437 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:45.091454 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d6343-6a7e-4031-82d2-553080786437 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:45.106561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-980d6343-6a7e-4031-82d2-553080786437 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:45.106561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:45.106561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:45.106561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:45.106561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:45.106561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:45.106561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:45.106561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:45.106561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:45.106561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:45.106561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:45.106561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:45.106561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:45.106561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:45.106561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:45.106561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:45.107549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:45.107549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:45.107549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:45.107549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:45.107549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:45.107549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:45.107549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:45.107549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:45.107549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:45.107549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:45.107549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:45.107549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:45.107549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:45.107549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:45.107549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:45.107549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:45.107549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:45.109630 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:45.109630 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:45.109630 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:45.109630 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 436/861] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:45.114381 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa5d05-44a3-447d-80d5-8078ab1a2790 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:45.215200 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:45.221028 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa5d05-44a3-447d-80d5-8078ab1a2790 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:45.256502 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f791a-916d-46f6-b3d4-3b55fa0e0b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:45.257132 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f791a-916d-46f6-b3d4-3b55fa0e0b65 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:45.257413 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f791a-916d-46f6-b3d4-3b55fa0e0b65 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:45.257687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f791a-916d-46f6-b3d4-3b55fa0e0b65 None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:45.258329 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474f791a-916d-46f6-b3d4-3b55fa0e0b65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:45.258651 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474f791a-916d-46f6-b3d4-3b55fa0e0b65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:45.260069 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474f791a-916d-46f6-b3d4-3b55fa0e0b65 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:45.273125 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 426/862] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:45.278890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8c1c9-6559-4768-946d-b06f387401b4 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:45.279034 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f791a-916d-46f6-b3d4-3b55fa0e0b65 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:45.279217 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8c1c9-6559-4768-946d-b06f387401b4 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:45.279483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8c1c9-6559-4768-946d-b06f387401b4 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:45.281580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 437/863] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:45.286513 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5570944-2fcc-4906-966a-ec3c1c3fdbaa None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:45.286791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5570944-2fcc-4906-966a-ec3c1c3fdbaa None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:45.287064 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5570944-2fcc-4906-966a-ec3c1c3fdbaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:45.305008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-f5570944-2fcc-4906-966a-ec3c1c3fdbaa None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:45.306158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f5570944-2fcc-4906-966a-ec3c1c3fdbaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:45.382986 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f791a-916d-46f6-b3d4-3b55fa0e0b65 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:45.388838 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 438/864] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 00:35:45.418379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7aa8d-608b-4a5f-ae66-151fbbddab0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:45.419019 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7aa8d-608b-4a5f-ae66-151fbbddab0d None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:45.419299 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7aa8d-608b-4a5f-ae66-151fbbddab0d None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:45.419574 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7aa8d-608b-4a5f-ae66-151fbbddab0d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:45.428445 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f7aa8d-608b-4a5f-ae66-151fbbddab0d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:45.429913 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f7aa8d-608b-4a5f-ae66-151fbbddab0d None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:45.442130 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-a5f7aa8d-608b-4a5f-ae66-151fbbddab0d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 00:35:45.442130 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:45.442130 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:45.442130 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:45.442130 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:45.442130 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:45.442130 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:45.442130 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:45.442130 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:45.442130 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:45.442130 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:45.442130 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:45.442130 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:35:45.442130 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:35:45.442130 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:35:45.442130 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:35:45.443078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:45.443078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:45.443078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:45.443078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:45.443078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:45.443078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:45.443078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:45.443078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:45.443078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:45.443078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:45.443078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:45.443078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:45.443078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:45.443078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:45.443078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:45.443078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:45.443078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:45.443078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:45.444755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:45.444755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:45.444755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:35:45.444755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:35:45.444755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:35:45.444755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:35:45.444755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:35:45.444755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:35:45.444755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 00:35:45.444755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:45.444755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 427/865] 10.222.0.59 () {56 vars in 1117 bytes} [Fri Aug 26 00:35:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:45.449210 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5570944-2fcc-4906-966a-ec3c1c3fdbaa None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:45.512975 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:45.555217 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5570944-2fcc-4906-966a-ec3c1c3fdbaa None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:45.591573 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a75a2c-70d3-4603-a3e2-387de864bac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:45.592606 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a75a2c-70d3-4603-a3e2-387de864bac5 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:45.593106 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a75a2c-70d3-4603-a3e2-387de864bac5 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:45.593632 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a75a2c-70d3-4603-a3e2-387de864bac5 None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:45.594590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a75a2c-70d3-4603-a3e2-387de864bac5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:45.595113 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a75a2c-70d3-4603-a3e2-387de864bac5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:45.596775 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a75a2c-70d3-4603-a3e2-387de864bac5 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:45.615163 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 439/866] 10.222.0.59 () {56 vars in 1149 bytes} [Fri Aug 26 00:35:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:45.624580 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7aa8d-608b-4a5f-ae66-151fbbddab0d None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:45.644852 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:45.683653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a75a2c-70d3-4603-a3e2-387de864bac5 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:45.736457 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7aa8d-608b-4a5f-ae66-151fbbddab0d None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:45.774422 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf16e5-2654-4c47-bfb4-77e30d63b63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:45.774951 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf16e5-2654-4c47-bfb4-77e30d63b63f None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:45.775234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf16e5-2654-4c47-bfb4-77e30d63b63f None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:45.775566 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf16e5-2654-4c47-bfb4-77e30d63b63f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:45.789384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a75a2c-70d3-4603-a3e2-387de864bac5 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:45.810662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:45.830278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c648f-1736-49e6-a8bf-e3f18806cf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:45.831797 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c648f-1736-49e6-a8bf-e3f18806cf7f None admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:45.832662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c648f-1736-49e6-a8bf-e3f18806cf7f None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:45.833470 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c648f-1736-49e6-a8bf-e3f18806cf7f None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:45.834982 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c648f-1736-49e6-a8bf-e3f18806cf7f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:45.837037 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c648f-1736-49e6-a8bf-e3f18806cf7f None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:45.854955 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf16e5-2654-4c47-bfb4-77e30d63b63f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:45.856420 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf16e5-2654-4c47-bfb4-77e30d63b63f None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:45.873943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:45.917676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 440/867] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Aug 26 00:35:45 2022] POST /identity/v3/services => generated 251 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:35:45.919227 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafecf3f-f39b-43f5-ab90-b2cfdf9ccddf None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:45.919509 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafecf3f-f39b-43f5-ab90-b2cfdf9ccddf None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:45.919843 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafecf3f-f39b-43f5-ab90-b2cfdf9ccddf None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:45.921445 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 441/868] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:45.927372 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc896b36-db31-48a9-83a1-73eb4e922cbc None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:45.927646 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc896b36-db31-48a9-83a1-73eb4e922cbc None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:45.927920 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc896b36-db31-48a9-83a1-73eb4e922cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:45.947898 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-dc896b36-db31-48a9-83a1-73eb4e922cbc None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:45.949785 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-dc896b36-db31-48a9-83a1-73eb4e922cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:45.970852 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 428/869] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:35:45 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 00:35:45.977598 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf16e5-2654-4c47-bfb4-77e30d63b63f None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:46.045305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 442/870] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:45 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:35:46.109140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf16e5-2654-4c47-bfb4-77e30d63b63f None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:46.126212 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:46.144901 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934c1a2-23b0-47ce-bce1-beb0a05e80b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:46.145536 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934c1a2-23b0-47ce-bce1-beb0a05e80b7 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:46.145890 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934c1a2-23b0-47ce-bce1-beb0a05e80b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:46.146188 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934c1a2-23b0-47ce-bce1-beb0a05e80b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:46.157594 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c934c1a2-23b0-47ce-bce1-beb0a05e80b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:46.159047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c934c1a2-23b0-47ce-bce1-beb0a05e80b7 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:46.174139 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-c934c1a2-23b0-47ce-bce1-beb0a05e80b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:46.174139 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:46.174139 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:46.174139 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:46.174139 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:46.174139 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:46.174139 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:46.174139 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:46.174139 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:46.174139 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:46.174139 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:46.174139 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:46.174139 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:46.174139 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:46.174139 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:46.174139 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:46.175702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:46.175702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:46.175702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:46.175702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:46.175702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:46.175702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:46.175702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:46.175702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:46.175702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:46.175702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:46.175702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:46.175702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:46.175702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:46.175702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:46.175702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:46.175702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:46.175702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:46.177303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:46.177303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:46.177303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:46.177303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 429/871] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:46.184543 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc896b36-db31-48a9-83a1-73eb4e922cbc None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:46.296521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc896b36-db31-48a9-83a1-73eb4e922cbc None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:46.304211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:46.310722 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934c1a2-23b0-47ce-bce1-beb0a05e80b7 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:46.333184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fca51d-9da7-4afb-a204-1898e324b7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:46.333926 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fca51d-9da7-4afb-a204-1898e324b7c1 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:46.334211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fca51d-9da7-4afb-a204-1898e324b7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:46.334491 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fca51d-9da7-4afb-a204-1898e324b7c1 None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:46.335150 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fca51d-9da7-4afb-a204-1898e324b7c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:46.335461 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fca51d-9da7-4afb-a204-1898e324b7c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:46.336984 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fca51d-9da7-4afb-a204-1898e324b7c1 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:46.351816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 443/872] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:46.357522 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fca51d-9da7-4afb-a204-1898e324b7c1 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:46.439973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934c1a2-23b0-47ce-bce1-beb0a05e80b7 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:46.474543 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fca51d-9da7-4afb-a204-1898e324b7c1 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:46.485359 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7759bf9-49a5-453b-9708-ff2735551f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:46.485749 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7759bf9-49a5-453b-9708-ff2735551f79 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:46.486029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7759bf9-49a5-453b-9708-ff2735551f79 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:46.486299 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7759bf9-49a5-453b-9708-ff2735551f79 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:46.486892 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7759bf9-49a5-453b-9708-ff2735551f79 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:46.488812 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7759bf9-49a5-453b-9708-ff2735551f79 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:46.501254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-f7759bf9-49a5-453b-9708-ff2735551f79 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 00:35:46.501254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:46.501254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 00:35:46.501254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 00:35:46.501254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 00:35:46.501254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 00:35:46.501254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 00:35:46.501254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:46.501254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 00:35:46.501254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:46.501254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:35:46.501254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:46.501254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:46.501254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:46.501254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:46.501254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:46.501254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:46.502304 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:46.502304 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:46.502304 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:46.502304 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:46.502304 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:46.502304 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:46.502304 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 00:35:46.502304 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 00:35:46.502304 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 00:35:46.502304 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 00:35:46.502304 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:46.502304 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:46.502304 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:46.502304 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:46.502304 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:46.502304 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:46.503219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:46.503219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:46.503219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:46.503219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:46.503219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:46.503219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:46.503219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:46.503219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:46.503219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:46.503219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:46.503219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 00:35:46.503219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:46.503219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 00:35:46.503219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:46.503219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 430/873] 10.222.0.59 () {56 vars in 1126 bytes} [Fri Aug 26 00:35:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:46.508513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7759bf9-49a5-453b-9708-ff2735551f79 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:46.523789 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa02be3-2059-4398-81fe-f9dd1bc88b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:46.524625 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa02be3-2059-4398-81fe-f9dd1bc88b6b None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:46.524971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa02be3-2059-4398-81fe-f9dd1bc88b6b None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:46.525310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa02be3-2059-4398-81fe-f9dd1bc88b6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:46.536146 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa02be3-2059-4398-81fe-f9dd1bc88b6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:46.538273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa02be3-2059-4398-81fe-f9dd1bc88b6b None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:46.548224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:46.556295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-9aa02be3-2059-4398-81fe-f9dd1bc88b6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:46.556295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:46.556295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:46.556295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:46.556295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:46.556295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:46.556295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:46.556295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:46.556295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:46.556295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:46.556295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:46.556295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:46.556295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:35:46.556295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:35:46.556295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:35:46.564169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:35:46.564169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:46.564169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:46.564169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:46.564169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:46.564169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:46.564169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:46.564169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:46.564169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:46.564169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:46.564169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:46.564169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:46.564169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:46.564169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:46.564169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:46.564169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:46.564169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:46.565120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:35:46.565120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:35:46.565120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:35:46.565120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:35:46.565120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:35:46.565120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:35:46.565120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:46.565120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:46.565120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 444/874] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:46.565120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa02be3-2059-4398-81fe-f9dd1bc88b6b None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:46.597034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:46.644897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7759bf9-49a5-453b-9708-ff2735551f79 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:46.676172 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa02be3-2059-4398-81fe-f9dd1bc88b6b None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:46.690612 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47498fb-4133-449f-b27d-d218af742aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:46.691229 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47498fb-4133-449f-b27d-d218af742aa6 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:46.691508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47498fb-4133-449f-b27d-d218af742aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:46.691780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47498fb-4133-449f-b27d-d218af742aa6 None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:46.692690 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47498fb-4133-449f-b27d-d218af742aa6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:46.692996 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47498fb-4133-449f-b27d-d218af742aa6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:46.694980 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47498fb-4133-449f-b27d-d218af742aa6 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:46.700872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 431/875] 10.222.0.59 () {56 vars in 1122 bytes} [Fri Aug 26 00:35:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:46.706708 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47498fb-4133-449f-b27d-d218af742aa6 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:46.720957 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3cf75-74f0-47f1-bed6-1072dbbb2d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:46.721570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3cf75-74f0-47f1-bed6-1072dbbb2d04 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:46.721852 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3cf75-74f0-47f1-bed6-1072dbbb2d04 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:46.722128 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3cf75-74f0-47f1-bed6-1072dbbb2d04 None admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:46.722826 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3cf75-74f0-47f1-bed6-1072dbbb2d04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:46.723135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3cf75-74f0-47f1-bed6-1072dbbb2d04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:46.724593 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3cf75-74f0-47f1-bed6-1072dbbb2d04 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:46.741841 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 445/876] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:35:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:46.742730 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3cf75-74f0-47f1-bed6-1072dbbb2d04 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:46.810040 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:46.817264 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47498fb-4133-449f-b27d-d218af742aa6 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:46.831642 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:46.847866 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3cf75-74f0-47f1-bed6-1072dbbb2d04 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:46.852399 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969d252-834c-4614-93e1-bdd4461c6589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:46.853088 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969d252-834c-4614-93e1-bdd4461c6589 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:46.853374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969d252-834c-4614-93e1-bdd4461c6589 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:46.853721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969d252-834c-4614-93e1-bdd4461c6589 None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:46.854363 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969d252-834c-4614-93e1-bdd4461c6589 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:46.854666 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969d252-834c-4614-93e1-bdd4461c6589 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:46.856028 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969d252-834c-4614-93e1-bdd4461c6589 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:46.861760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 432/877] 10.222.0.59 () {56 vars in 1122 bytes} [Fri Aug 26 00:35:46 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:46.893264 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6f10d-d115-4a83-a1b7-29a981212d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:46.893966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b459058-2a4e-4bb7-8032-2c0dcc2c1e1f None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:46.894140 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6f10d-d115-4a83-a1b7-29a981212d26 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:46.894327 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b459058-2a4e-4bb7-8032-2c0dcc2c1e1f None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:46.894439 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6f10d-d115-4a83-a1b7-29a981212d26 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:46.894517 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b459058-2a4e-4bb7-8032-2c0dcc2c1e1f None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:46.894697 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6f10d-d115-4a83-a1b7-29a981212d26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:46.896055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e6f10d-d115-4a83-a1b7-29a981212d26 None admin] RBAC: Adding query filter params (role.id=bce78c7e0a534135a820de7f3588aa4d, scope.project.id=fde7ac3ed472433ab3bba56e5e091510, user.id=bbd9bbb083ab4f5e84e135ec5103cdae) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:46.896175 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 433/878] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:35:46.896251 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e6f10d-d115-4a83-a1b7-29a981212d26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:46.897835 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e6f10d-d115-4a83-a1b7-29a981212d26 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:46.902651 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94778b6-d24f-4636-8995-ecc269194a4a None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:46.903057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94778b6-d24f-4636-8995-ecc269194a4a None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:46.903190 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94778b6-d24f-4636-8995-ecc269194a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:46.906480 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 446/879] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:35:46 2022] GET /identity/v3/role_assignments?user.id=bbd9bbb083ab4f5e84e135ec5103cdae&scope.project.id=fde7ac3ed472433ab3bba56e5e091510&role.id=bce78c7e0a534135a820de7f3588aa4d => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:46.924413 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-f94778b6-d24f-4636-8995-ecc269194a4a None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:46.925714 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-f94778b6-d24f-4636-8995-ecc269194a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:47.010149 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 434/880] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:46 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 00:35:47.288441 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6f10d-d115-4a83-a1b7-29a981212d26 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:47.393346 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6f10d-d115-4a83-a1b7-29a981212d26 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:47.428629 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a1da29-4bd7-40fb-a424-39d403fe7a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:47.429250 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1da29-4bd7-40fb-a424-39d403fe7a06 None admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:47.429529 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1da29-4bd7-40fb-a424-39d403fe7a06 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:47.429939 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1da29-4bd7-40fb-a424-39d403fe7a06 None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:47.430472 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a1da29-4bd7-40fb-a424-39d403fe7a06 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:47.431848 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a1da29-4bd7-40fb-a424-39d403fe7a06 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:47.547606 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 447/881] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Aug 26 00:35:47 2022] POST /identity/v3/services => generated 269 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:35:48.056086 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a78eb1-6db8-4b38-b01a-e2e302cc77bc None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:48.057167 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a78eb1-6db8-4b38-b01a-e2e302cc77bc None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:48.057663 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a78eb1-6db8-4b38-b01a-e2e302cc77bc None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:48.059732 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 435/882] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:48.065476 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c517d4e-a399-44f2-ad3d-7611f24e95c8 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:48.066326 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c517d4e-a399-44f2-ad3d-7611f24e95c8 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:48.066822 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c517d4e-a399-44f2-ad3d-7611f24e95c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:48.086100 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-6c517d4e-a399-44f2-ad3d-7611f24e95c8 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:48.087825 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-6c517d4e-a399-44f2-ad3d-7611f24e95c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:48.090981 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:48.171357 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 448/883] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:35:48.466424 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a78eb1-6db8-4b38-b01a-e2e302cc77bc None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:48.566031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:48.577896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a78eb1-6db8-4b38-b01a-e2e302cc77bc None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:48.612109 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9fddf-258a-4a27-ac88-bf8608efbe3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:48.612779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9fddf-258a-4a27-ac88-bf8608efbe3c None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:48.613095 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9fddf-258a-4a27-ac88-bf8608efbe3c None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:48.613314 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9fddf-258a-4a27-ac88-bf8608efbe3c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:48.614384 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9fddf-258a-4a27-ac88-bf8608efbe3c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:48.616122 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9fddf-258a-4a27-ac88-bf8608efbe3c None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:48.625803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-a3c9fddf-258a-4a27-ac88-bf8608efbe3c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 00:35:48.625803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:48.625803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 00:35:48.625803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 00:35:48.625803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 00:35:48.625803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 00:35:48.625803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 00:35:48.625803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:48.625803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 00:35:48.625803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:48.625803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:35:48.625803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:48.625803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:48.625803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:48.625803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:48.625803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:48.625803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:48.626824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:48.626824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:48.626824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:48.626824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:48.626824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:48.626824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:48.626824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 00:35:48.626824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 00:35:48.626824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 00:35:48.626824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 00:35:48.626824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:48.626824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:48.626824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:48.626824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:48.626824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:48.626824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:48.628076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:48.628076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:48.628076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:48.628076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:48.628076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:48.628076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:48.628076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:48.628076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:48.628076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:48.628076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:48.628076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 00:35:48.628076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:48.628076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 00:35:48.628076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:48.628076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 436/884] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Aug 26 00:35:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:48.635517 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c517d4e-a399-44f2-ad3d-7611f24e95c8 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:48.753105 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c517d4e-a399-44f2-ad3d-7611f24e95c8 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:48.792913 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e7fc4d-69f0-4d1a-85c4-60cb5aaebf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:48.793547 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e7fc4d-69f0-4d1a-85c4-60cb5aaebf39 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:48.793842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e7fc4d-69f0-4d1a-85c4-60cb5aaebf39 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:48.794102 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e7fc4d-69f0-4d1a-85c4-60cb5aaebf39 None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:48.794220 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b83193-a015-4a73-8abf-7a09d0d4c2b5 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:48.794476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b83193-a015-4a73-8abf-7a09d0d4c2b5 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:48.794749 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e7fc4d-69f0-4d1a-85c4-60cb5aaebf39 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:48.794852 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b83193-a015-4a73-8abf-7a09d0d4c2b5 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:48.795039 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e7fc4d-69f0-4d1a-85c4-60cb5aaebf39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:48.796963 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e7fc4d-69f0-4d1a-85c4-60cb5aaebf39 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:48.797077 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 437/885] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:48.801851 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1bb1f-f127-4d82-9363-3c9f1d05d4d0 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:48.802077 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 449/886] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Aug 26 00:35:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:48.802182 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1bb1f-f127-4d82-9363-3c9f1d05d4d0 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:48.802394 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1bb1f-f127-4d82-9363-3c9f1d05d4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:48.802575 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:48.808429 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e7fc4d-69f0-4d1a-85c4-60cb5aaebf39 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:48.826573 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-81d1bb1f-f127-4d82-9363-3c9f1d05d4d0 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:48.828107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-81d1bb1f-f127-4d82-9363-3c9f1d05d4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:48.908813 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 438/887] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 00:35:48.932775 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e7fc4d-69f0-4d1a-85c4-60cb5aaebf39 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:48.969201 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5cbbc-719a-448e-9047-a8195d6c57a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:48.969819 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5cbbc-719a-448e-9047-a8195d6c57a0 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:48.970101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5cbbc-719a-448e-9047-a8195d6c57a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:48.970377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5cbbc-719a-448e-9047-a8195d6c57a0 None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:48.971488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5cbbc-719a-448e-9047-a8195d6c57a0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:48.971488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5cbbc-719a-448e-9047-a8195d6c57a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:48.972706 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5cbbc-719a-448e-9047-a8195d6c57a0 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:48.982776 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 450/888] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Aug 26 00:35:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:35:48.984706 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d1bb1f-f127-4d82-9363-3c9f1d05d4d0 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:49.074641 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:49.076088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79e909-a467-4af3-8be2-138ae46f3b2f None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:49.076501 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79e909-a467-4af3-8be2-138ae46f3b2f None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:49.076671 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79e909-a467-4af3-8be2-138ae46f3b2f None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:49.078210 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 451/889] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:49.083490 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446a0d1-e080-4a29-8313-5753a22c47b9 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:49.084032 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446a0d1-e080-4a29-8313-5753a22c47b9 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:49.084540 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446a0d1-e080-4a29-8313-5753a22c47b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:49.099388 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d1bb1f-f127-4d82-9363-3c9f1d05d4d0 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:49.103996 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-a446a0d1-e080-4a29-8313-5753a22c47b9 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:49.105528 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a446a0d1-e080-4a29-8313-5753a22c47b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:49.143635 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288ee9b-99f3-4d05-9355-b880bd8e7fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:49.144253 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288ee9b-99f3-4d05-9355-b880bd8e7fc7 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:49.144563 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288ee9b-99f3-4d05-9355-b880bd8e7fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:49.144827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288ee9b-99f3-4d05-9355-b880bd8e7fc7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:49.145537 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c288ee9b-99f3-4d05-9355-b880bd8e7fc7 None admin] RBAC: Adding query filter params (interface=public, service_id=7f70fdfba269476e884199d1129e9ade, region_id=RegionOne) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:49.145857 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c288ee9b-99f3-4d05-9355-b880bd8e7fc7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:49.148448 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c288ee9b-99f3-4d05-9355-b880bd8e7fc7 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:49.160110 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 439/890] 10.222.0.59 () {56 vars in 1251 bytes} [Fri Aug 26 00:35:48 2022] GET /identity/v3/endpoints?service_id=7f70fdfba269476e884199d1129e9ade&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:49.161404 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288ee9b-99f3-4d05-9355-b880bd8e7fc7 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:49.180745 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:49.189967 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 452/891] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:49 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 00:35:49.197412 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446a0d1-e080-4a29-8313-5753a22c47b9 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:49.273711 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288ee9b-99f3-4d05-9355-b880bd8e7fc7 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:49.309963 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446a0d1-e080-4a29-8313-5753a22c47b9 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:49.310205 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a10657a-d52a-4864-adaf-d0f6c4a7d38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:49.310807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a10657a-d52a-4864-adaf-d0f6c4a7d38d None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:49.311225 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a10657a-d52a-4864-adaf-d0f6c4a7d38d None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:49.311652 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a10657a-d52a-4864-adaf-d0f6c4a7d38d None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:49.312355 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a10657a-d52a-4864-adaf-d0f6c4a7d38d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:49.313996 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a10657a-d52a-4864-adaf-d0f6c4a7d38d None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:49.324581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 440/892] 10.222.0.59 () {56 vars in 1087 bytes} [Fri Aug 26 00:35:49 2022] GET /identity/v3/services => generated 1268 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:35:49.350008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96996188-d442-4295-85fc-8360f89d1207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:49.350881 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96996188-d442-4295-85fc-8360f89d1207 None admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:49.351310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96996188-d442-4295-85fc-8360f89d1207 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:49.351783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96996188-d442-4295-85fc-8360f89d1207 None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:49.352634 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96996188-d442-4295-85fc-8360f89d1207 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:49.354184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96996188-d442-4295-85fc-8360f89d1207 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:49.437159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 453/893] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Aug 26 00:35:49 2022] POST /identity/v3/services => generated 241 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:35:49.466068 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a10657a-d52a-4864-adaf-d0f6c4a7d38d None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:49.484572 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:49.554316 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae76d3-f331-4a43-8dbd-aad989f930d2 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:49.554600 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae76d3-f331-4a43-8dbd-aad989f930d2 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:49.554877 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae76d3-f331-4a43-8dbd-aad989f930d2 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:49.556545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 454/894] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:35:49.562089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4232fb-3338-4cd9-8967-de7837965c0b None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:49.562371 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4232fb-3338-4cd9-8967-de7837965c0b None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:49.562647 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4232fb-3338-4cd9-8967-de7837965c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:49.589739 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a10657a-d52a-4864-adaf-d0f6c4a7d38d None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:49.589948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-ce4232fb-3338-4cd9-8967-de7837965c0b None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:49.590988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-ce4232fb-3338-4cd9-8967-de7837965c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:49.640210 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b6948-11a6-4e16-88f3-c08c591f9d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:49.640799 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b6948-11a6-4e16-88f3-c08c591f9d3f None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:49.641076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b6948-11a6-4e16-88f3-c08c591f9d3f None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:49.641342 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b6948-11a6-4e16-88f3-c08c591f9d3f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:49.651786 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b6948-11a6-4e16-88f3-c08c591f9d3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:49.658018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:49.659534 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-d57b6948-11a6-4e16-88f3-c08c591f9d3f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 00:35:49.659534 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:49.659534 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:49.659534 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:49.659534 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:49.659534 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:49.659534 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:49.659534 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:49.659534 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:49.659534 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:49.659534 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:49.659534 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:49.659534 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:49.659534 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:49.659534 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:49.659534 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:49.660505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:49.660505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:49.660505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:49.660505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:49.660505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:49.660505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:49.660505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:49.660505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:49.660505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:49.660505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:49.660505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:49.660505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:49.660505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:49.660505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:49.660505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:49.660505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:49.661387 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:49.661387 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:49.661387 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:49.661387 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:49.661387 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:49.661387 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:49.661387 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:49.661387 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:49.661387 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:49.661387 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:49.661387 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 00:35:49.661387 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:49.661387 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 441/895] 10.222.0.59 () {56 vars in 1120 bytes} [Fri Aug 26 00:35:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:49.671145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b6948-11a6-4e16-88f3-c08c591f9d3f None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:49.682178 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 455/896] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 00:35:49.745555 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:49.781996 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b6948-11a6-4e16-88f3-c08c591f9d3f None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:49.819018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f0b00-dd67-481e-ad84-db1716ab1000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:49.819636 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f0b00-dd67-481e-ad84-db1716ab1000 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:49.819921 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f0b00-dd67-481e-ad84-db1716ab1000 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:49.820199 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f0b00-dd67-481e-ad84-db1716ab1000 None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:49.824108 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f0b00-dd67-481e-ad84-db1716ab1000 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:49.824108 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f0b00-dd67-481e-ad84-db1716ab1000 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:49.824108 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f0b00-dd67-481e-ad84-db1716ab1000 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:49.832443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 442/897] 10.222.0.59 () {56 vars in 1115 bytes} [Fri Aug 26 00:35:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:35:49.837861 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4232fb-3338-4cd9-8967-de7837965c0b None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:49.948730 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4232fb-3338-4cd9-8967-de7837965c0b None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:49.977696 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f0b00-dd67-481e-ad84-db1716ab1000 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:49.984734 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052232c-4fb3-4ce7-88f3-e97c236a8aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:49.985378 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052232c-4fb3-4ce7-88f3-e97c236a8aa3 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:49.985655 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052232c-4fb3-4ce7-88f3-e97c236a8aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:49.985925 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052232c-4fb3-4ce7-88f3-e97c236a8aa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:49.998021 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052232c-4fb3-4ce7-88f3-e97c236a8aa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:49.999507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052232c-4fb3-4ce7-88f3-e97c236a8aa3 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:50.019247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-1052232c-4fb3-4ce7-88f3-e97c236a8aa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:50.019247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:50.019247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:50.019247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:50.019247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:50.019247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:50.019247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:50.019247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:50.019247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:50.019247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:50.019247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:50.019247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:50.019247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:50.019247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:50.019247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:50.019247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:50.021550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:50.021550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:50.021550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:50.021550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:50.021550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:50.021550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:50.021550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:50.021550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:50.021550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:50.021550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:50.021550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:50.021550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:50.021550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:50.021550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:50.021550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:50.021550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:50.021550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:50.022470 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:50.022470 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:50.022470 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:50.024646 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 456/898] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:50.029747 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052232c-4fb3-4ce7-88f3-e97c236a8aa3 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:50.085671 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:50.111477 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f0b00-dd67-481e-ad84-db1716ab1000 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:50.150061 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:50.159164 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51da3247-4d4b-4910-b88b-0f09e9017202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:50.159995 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da3247-4d4b-4910-b88b-0f09e9017202 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:50.160141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052232c-4fb3-4ce7-88f3-e97c236a8aa3 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:50.160587 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da3247-4d4b-4910-b88b-0f09e9017202 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:50.161010 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da3247-4d4b-4910-b88b-0f09e9017202 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:50.161882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51da3247-4d4b-4910-b88b-0f09e9017202 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:50.164323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51da3247-4d4b-4910-b88b-0f09e9017202 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:50.175173 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-51da3247-4d4b-4910-b88b-0f09e9017202 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 00:35:50.175173 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:50.175173 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 00:35:50.175173 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 00:35:50.175173 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 00:35:50.175173 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 00:35:50.175173 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 00:35:50.175173 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:50.175173 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 00:35:50.175173 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:50.175173 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:35:50.175173 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:50.175173 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:50.175173 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:50.175173 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:50.175173 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:50.176391 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:50.176391 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:50.176391 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:50.176391 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:50.176391 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:50.176391 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:50.176391 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:50.176391 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 00:35:50.176391 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 00:35:50.176391 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 00:35:50.176391 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 00:35:50.176391 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:50.176391 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:50.176391 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:50.176391 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:50.176391 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:50.176391 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:50.177585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:50.177585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:50.177585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:50.177585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:50.177585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:50.177585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:50.177585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:50.177585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:50.177585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:50.177585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:50.177585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 00:35:50.177585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:50.177585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 00:35:50.177585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:50.179337 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 443/899] 10.222.0.59 () {56 vars in 1132 bytes} [Fri Aug 26 00:35:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:50.188205 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51da3247-4d4b-4910-b88b-0f09e9017202 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:50.211129 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e996c-9ada-4adf-8fec-8086066e3403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:50.211634 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e996c-9ada-4adf-8fec-8086066e3403 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:50.211927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e996c-9ada-4adf-8fec-8086066e3403 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:50.212206 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e996c-9ada-4adf-8fec-8086066e3403 None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:50.212886 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e996c-9ada-4adf-8fec-8086066e3403 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:50.213300 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e996c-9ada-4adf-8fec-8086066e3403 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:50.214786 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e996c-9ada-4adf-8fec-8086066e3403 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:50.232200 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 457/900] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:50.233853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e996c-9ada-4adf-8fec-8086066e3403 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:50.312239 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51da3247-4d4b-4910-b88b-0f09e9017202 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:50.351507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:50.358767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b7749-57b8-47a4-baea-9023cb0c5acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:50.358917 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e996c-9ada-4adf-8fec-8086066e3403 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:50.359722 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b7749-57b8-47a4-baea-9023cb0c5acc None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:50.362908 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b7749-57b8-47a4-baea-9023cb0c5acc None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:50.362908 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b7749-57b8-47a4-baea-9023cb0c5acc None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:50.362908 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b7749-57b8-47a4-baea-9023cb0c5acc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:50.362908 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b7749-57b8-47a4-baea-9023cb0c5acc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:50.364235 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b7749-57b8-47a4-baea-9023cb0c5acc None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:50.370646 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 444/901] 10.222.0.59 () {56 vars in 1126 bytes} [Fri Aug 26 00:35:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:35:50.376339 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b7749-57b8-47a4-baea-9023cb0c5acc None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:50.408035 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bcc99d-b773-4b0b-8bcc-13b1a26f3236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:50.408724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bcc99d-b773-4b0b-8bcc-13b1a26f3236 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:50.409007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bcc99d-b773-4b0b-8bcc-13b1a26f3236 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:50.409280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bcc99d-b773-4b0b-8bcc-13b1a26f3236 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:50.419057 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bcc99d-b773-4b0b-8bcc-13b1a26f3236 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:50.420602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bcc99d-b773-4b0b-8bcc-13b1a26f3236 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:50.437436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-99bcc99d-b773-4b0b-8bcc-13b1a26f3236 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 00:35:50.437436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:50.437436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:50.437436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:50.437436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:50.437436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:50.437436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:50.437436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:50.437436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:50.437436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:50.437436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:50.437436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:50.437436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:35:50.437436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:35:50.437436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:35:50.437436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:35:50.438584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:50.438584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:50.438584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:50.438584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:50.438584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:50.438584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:50.438584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:50.438584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:50.438584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:50.438584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:50.438584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:50.438584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:50.438584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:50.438584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:50.438584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:50.438584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:50.438584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:50.438584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:50.439763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:50.439763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:50.439763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:35:50.439763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:35:50.439763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:35:50.439763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:35:50.439763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:35:50.439763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:35:50.439763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 00:35:50.439763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:50.445916 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 458/902] 10.222.0.59 () {56 vars in 1117 bytes} [Fri Aug 26 00:35:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:50.446920 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bcc99d-b773-4b0b-8bcc-13b1a26f3236 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:50.460644 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:50.512752 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b7749-57b8-47a4-baea-9023cb0c5acc None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:50.551602 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1a62c-6b4c-462f-884a-084563fbd427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:50.552399 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1a62c-6b4c-462f-884a-084563fbd427 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:50.552802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1a62c-6b4c-462f-884a-084563fbd427 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:50.553174 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1a62c-6b4c-462f-884a-084563fbd427 None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:50.553911 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a1a62c-6b4c-462f-884a-084563fbd427 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:50.554334 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a1a62c-6b4c-462f-884a-084563fbd427 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:50.555790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a1a62c-6b4c-462f-884a-084563fbd427 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:50.566204 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 445/903] 10.222.0.59 () {56 vars in 1126 bytes} [Fri Aug 26 00:35:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:35:50.566795 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1a62c-6b4c-462f-884a-084563fbd427 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:50.584368 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bcc99d-b773-4b0b-8bcc-13b1a26f3236 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:50.627842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940ad1e-c7b8-44e1-a35d-dde756be2927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:50.628503 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940ad1e-c7b8-44e1-a35d-dde756be2927 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:50.628793 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940ad1e-c7b8-44e1-a35d-dde756be2927 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:50.629070 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940ad1e-c7b8-44e1-a35d-dde756be2927 None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:50.629830 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940ad1e-c7b8-44e1-a35d-dde756be2927 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:50.630132 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940ad1e-c7b8-44e1-a35d-dde756be2927 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:50.632380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940ad1e-c7b8-44e1-a35d-dde756be2927 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:50.659733 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 459/904] 10.222.0.59 () {56 vars in 1149 bytes} [Fri Aug 26 00:35:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:50.661609 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940ad1e-c7b8-44e1-a35d-dde756be2927 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:50.677972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:50.692994 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1a62c-6b4c-462f-884a-084563fbd427 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:50.716295 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:50.727648 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b7a623-48a5-45df-b841-160e1a006450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:50.730301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7a623-48a5-45df-b841-160e1a006450 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:50.730301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7a623-48a5-45df-b841-160e1a006450 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:50.730301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7a623-48a5-45df-b841-160e1a006450 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:50.730301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b7a623-48a5-45df-b841-160e1a006450 None admin] RBAC: Adding query filter params (interface=public, service_id=833908f4fd5b41a08b571ebc84a6c3c4, region_id=RegionOne) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:50.730301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b7a623-48a5-45df-b841-160e1a006450 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:50.731225 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b7a623-48a5-45df-b841-160e1a006450 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:50.740518 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 446/905] 10.222.0.59 () {56 vars in 1251 bytes} [Fri Aug 26 00:35:50 2022] GET /identity/v3/endpoints?service_id=833908f4fd5b41a08b571ebc84a6c3c4&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:35:50.743066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b7a623-48a5-45df-b841-160e1a006450 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:50.791056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940ad1e-c7b8-44e1-a35d-dde756be2927 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:50.848635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a3df1-473c-4263-b77b-a85ee81cb96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:50.849352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a3df1-473c-4263-b77b-a85ee81cb96c None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:50.849707 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a3df1-473c-4263-b77b-a85ee81cb96c None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:50.850335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a3df1-473c-4263-b77b-a85ee81cb96c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:50.874733 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b7a623-48a5-45df-b841-160e1a006450 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:50.918790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff619c-8cdd-4aa1-a208-87421579c30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:50.919383 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff619c-8cdd-4aa1-a208-87421579c30a None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:50.919756 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff619c-8cdd-4aa1-a208-87421579c30a None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:50.920174 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff619c-8cdd-4aa1-a208-87421579c30a None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:50.920817 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff619c-8cdd-4aa1-a208-87421579c30a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:50.922366 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff619c-8cdd-4aa1-a208-87421579c30a None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:50.936815 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 447/906] 10.222.0.59 () {56 vars in 1087 bytes} [Fri Aug 26 00:35:50 2022] GET /identity/v3/services => generated 1497 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 00:35:50.940897 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a3df1-473c-4263-b77b-a85ee81cb96c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:50.942406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a3df1-473c-4263-b77b-a85ee81cb96c None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:50.959961 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:51.056385 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 460/907] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:35:50 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 00:35:51.066494 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff619c-8cdd-4aa1-a208-87421579c30a None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:51.121288 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:51.175365 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff619c-8cdd-4aa1-a208-87421579c30a None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:51.210239 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ce284-256d-4dbf-a6bc-cd620f138a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:51.210872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ce284-256d-4dbf-a6bc-cd620f138a75 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:51.211150 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ce284-256d-4dbf-a6bc-cd620f138a75 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:51.211423 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ce284-256d-4dbf-a6bc-cd620f138a75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:51.222147 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ce284-256d-4dbf-a6bc-cd620f138a75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:51.223579 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ce284-256d-4dbf-a6bc-cd620f138a75 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:51.238563 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-ba2ce284-256d-4dbf-a6bc-cd620f138a75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:51.238563 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:51.238563 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:51.238563 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:51.238563 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:51.238563 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:51.238563 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:51.238563 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:51.238563 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:51.238563 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:51.238563 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:51.238563 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:51.238563 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:51.238563 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:51.238563 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:51.238563 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:51.239565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:51.239565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:51.239565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:51.239565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:51.239565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:51.239565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:51.239565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:51.239565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:51.239565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:51.239565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:51.239565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:51.239565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:51.239565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:51.239565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:51.239565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:51.239565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:51.239565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:51.240489 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:51.240489 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:51.240489 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:51.240489 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 448/908] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:51.245595 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a3df1-473c-4263-b77b-a85ee81cb96c None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:51.350235 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a3df1-473c-4263-b77b-a85ee81cb96c None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:51.365422 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:51.386038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5bdba4-9f74-48fb-abfa-f1064d0a77fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:51.386652 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bdba4-9f74-48fb-abfa-f1064d0a77fb None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:51.386927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bdba4-9f74-48fb-abfa-f1064d0a77fb None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:51.387196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bdba4-9f74-48fb-abfa-f1064d0a77fb None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:51.388014 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5bdba4-9f74-48fb-abfa-f1064d0a77fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:51.388314 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5bdba4-9f74-48fb-abfa-f1064d0a77fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:51.389779 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5bdba4-9f74-48fb-abfa-f1064d0a77fb None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:51.402111 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 461/909] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:51.408519 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ce284-256d-4dbf-a6bc-cd620f138a75 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:51.420396 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ff0ec-7620-46db-a7db-5a0385590c7a None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:51.421298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ff0ec-7620-46db-a7db-5a0385590c7a None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:51.421510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ff0ec-7620-46db-a7db-5a0385590c7a None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:51.423215 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 462/910] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:51.430789 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec67783-e9f5-413b-a5d7-8d0d4e4735eb None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:51.431068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec67783-e9f5-413b-a5d7-8d0d4e4735eb None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:51.431341 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec67783-e9f5-413b-a5d7-8d0d4e4735eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:51.450165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-5ec67783-e9f5-413b-a5d7-8d0d4e4735eb None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:51.451515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-5ec67783-e9f5-413b-a5d7-8d0d4e4735eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:51.532668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 463/911] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:35:51.534392 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510caf7-f53e-4487-8019-f74b39f57d7c None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:51.534924 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510caf7-f53e-4487-8019-f74b39f57d7c None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:51.535407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510caf7-f53e-4487-8019-f74b39f57d7c None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:51.537208 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 464/912] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:51.544106 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9c631-7eb0-4ff8-bbea-b5dae127a5a8 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:51.544677 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9c631-7eb0-4ff8-bbea-b5dae127a5a8 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:51.545179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9c631-7eb0-4ff8-bbea-b5dae127a5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:51.550243 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ce284-256d-4dbf-a6bc-cd620f138a75 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:51.551785 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:51.565512 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-acf9c631-7eb0-4ff8-bbea-b5dae127a5a8 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:51.567010 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-acf9c631-7eb0-4ff8-bbea-b5dae127a5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:51.590997 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d9978f-4105-4294-a60b-cd279f9a45ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:51.592906 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9978f-4105-4294-a60b-cd279f9a45ad None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:51.593711 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9978f-4105-4294-a60b-cd279f9a45ad None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:51.594292 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9978f-4105-4294-a60b-cd279f9a45ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:51.603416 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d9978f-4105-4294-a60b-cd279f9a45ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:51.604822 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d9978f-4105-4294-a60b-cd279f9a45ad None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:51.622508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-b3d9978f-4105-4294-a60b-cd279f9a45ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:51.622508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:51.622508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:51.622508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:51.622508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:51.622508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:51.622508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:51.622508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:51.622508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:51.622508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:51.622508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:51.622508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:51.622508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:35:51.622508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:35:51.622508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:35:51.623476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:35:51.623476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:51.623476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:51.623476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:51.623476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:51.623476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:51.623476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:51.623476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:51.623476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:51.623476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:51.623476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:51.623476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:51.623476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:51.623476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:51.623476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:51.623476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:51.623476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:51.625111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:35:51.625111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:35:51.625111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:35:51.625111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:35:51.625111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:35:51.625111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:35:51.625111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:51.625111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:51.625111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 449/913] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:51.629682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d9978f-4105-4294-a60b-cd279f9a45ad None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:51.652825 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 465/914] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:35:51.743268 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d9978f-4105-4294-a60b-cd279f9a45ad None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:51.780602 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda0e05-1fa7-409b-a700-8a00fdff7fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:51.781568 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda0e05-1fa7-409b-a700-8a00fdff7fe8 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:51.782054 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda0e05-1fa7-409b-a700-8a00fdff7fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:51.782531 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda0e05-1fa7-409b-a700-8a00fdff7fe8 None admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:51.783459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda0e05-1fa7-409b-a700-8a00fdff7fe8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:51.783993 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda0e05-1fa7-409b-a700-8a00fdff7fe8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:51.785691 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda0e05-1fa7-409b-a700-8a00fdff7fe8 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:51.798221 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 450/915] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:35:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:35:51.806299 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf9c631-7eb0-4ff8-bbea-b5dae127a5a8 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:51.818576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda0e05-1fa7-409b-a700-8a00fdff7fe8 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:51.840883 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:51.855385 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:51.918199 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf9c631-7eb0-4ff8-bbea-b5dae127a5a8 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:51.956048 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691de185-5e3c-4fdf-9066-d6d1de8bc1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:51.956709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda0e05-1fa7-409b-a700-8a00fdff7fe8 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:51.956848 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691de185-5e3c-4fdf-9066-d6d1de8bc1a8 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:51.957035 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691de185-5e3c-4fdf-9066-d6d1de8bc1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:51.957246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691de185-5e3c-4fdf-9066-d6d1de8bc1a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:51.957981 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691de185-5e3c-4fdf-9066-d6d1de8bc1a8 None admin] RBAC: Adding query filter params (role.id=91940e122e2f46e3818c5f11f4fca9e9, scope.project.id=fde7ac3ed472433ab3bba56e5e091510, user.id=bbd9bbb083ab4f5e84e135ec5103cdae) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:51.958297 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691de185-5e3c-4fdf-9066-d6d1de8bc1a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:51.959748 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691de185-5e3c-4fdf-9066-d6d1de8bc1a8 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:51.965495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 466/916] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:35:51 2022] GET /identity/v3/role_assignments?user.id=bbd9bbb083ab4f5e84e135ec5103cdae&scope.project.id=fde7ac3ed472433ab3bba56e5e091510&role.id=91940e122e2f46e3818c5f11f4fca9e9 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:51.966849 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691de185-5e3c-4fdf-9066-d6d1de8bc1a8 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:52.019875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76130a64-fcfb-4f1f-bcbf-b24d61944e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:52.021125 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76130a64-fcfb-4f1f-bcbf-b24d61944e69 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:52.021433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76130a64-fcfb-4f1f-bcbf-b24d61944e69 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:52.021718 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76130a64-fcfb-4f1f-bcbf-b24d61944e69 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:52.022331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76130a64-fcfb-4f1f-bcbf-b24d61944e69 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:52.023761 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76130a64-fcfb-4f1f-bcbf-b24d61944e69 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:52.040496 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-76130a64-fcfb-4f1f-bcbf-b24d61944e69 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 00:35:52.040496 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:52.040496 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 00:35:52.040496 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 00:35:52.040496 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 00:35:52.040496 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 00:35:52.040496 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 00:35:52.040496 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:52.040496 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 00:35:52.040496 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:52.040496 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:35:52.040496 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:52.040496 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:52.040496 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:52.040496 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:52.040496 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:52.040496 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:52.041884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:52.041884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:52.041884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:52.041884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:52.041884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:52.041884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:52.041884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 00:35:52.041884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 00:35:52.041884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 00:35:52.041884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 00:35:52.041884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:52.041884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:52.041884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:52.041884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:52.041884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:52.041884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:52.043005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:52.043005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:52.043005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:52.043005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:52.043005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:52.043005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:52.043005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:52.043005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:52.043005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:52.043005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:52.043005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 00:35:52.043005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:52.043005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 00:35:52.043005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:52.043005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 451/917] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Aug 26 00:35:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:52.045378 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76130a64-fcfb-4f1f-bcbf-b24d61944e69 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:52.101883 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:52.106685 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691de185-5e3c-4fdf-9066-d6d1de8bc1a8 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:52.148805 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e043ee-3200-49dd-92c6-9c5e7a6eeac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:52.149474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e043ee-3200-49dd-92c6-9c5e7a6eeac6 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:52.149755 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e043ee-3200-49dd-92c6-9c5e7a6eeac6 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:52.152499 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e043ee-3200-49dd-92c6-9c5e7a6eeac6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:52.152499 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e043ee-3200-49dd-92c6-9c5e7a6eeac6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:52.152905 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e043ee-3200-49dd-92c6-9c5e7a6eeac6 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:52.164415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:52.166089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-40e043ee-3200-49dd-92c6-9c5e7a6eeac6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 00:35:52.166089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:52.166089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 00:35:52.166089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 00:35:52.166089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 00:35:52.166089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 00:35:52.166089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 00:35:52.166089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:52.166089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 00:35:52.166089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:52.166089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:35:52.166089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:52.166089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:52.166089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:52.166089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:52.166089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:52.166089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:52.167090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:52.167090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:52.167090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:52.167090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:52.167090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:52.167090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:52.167090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 00:35:52.167090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 00:35:52.167090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 00:35:52.167090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 00:35:52.167090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:52.167090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:52.167090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:52.167090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:52.167090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:52.167090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:52.167949 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:52.167949 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:52.167949 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:52.167949 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:52.167949 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:52.167949 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:52.167949 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:52.167949 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:52.167949 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:52.167949 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:52.167949 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 00:35:52.167949 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:52.167949 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 00:35:52.167949 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:52.167949 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 467/918] 10.222.0.59 () {56 vars in 1126 bytes} [Fri Aug 26 00:35:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:52.174148 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e043ee-3200-49dd-92c6-9c5e7a6eeac6 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:52.187036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76130a64-fcfb-4f1f-bcbf-b24d61944e69 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:52.221566 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1129ef-5fb7-46fd-a0b8-cbd35ea29888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:52.222335 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1129ef-5fb7-46fd-a0b8-cbd35ea29888 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:52.222711 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1129ef-5fb7-46fd-a0b8-cbd35ea29888 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:52.223085 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1129ef-5fb7-46fd-a0b8-cbd35ea29888 None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:52.223814 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1129ef-5fb7-46fd-a0b8-cbd35ea29888 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:52.224221 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1129ef-5fb7-46fd-a0b8-cbd35ea29888 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:52.225718 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1129ef-5fb7-46fd-a0b8-cbd35ea29888 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:52.232259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 452/919] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Aug 26 00:35:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:52.238926 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1129ef-5fb7-46fd-a0b8-cbd35ea29888 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:52.297165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e043ee-3200-49dd-92c6-9c5e7a6eeac6 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:52.334172 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83acedd-b9b9-4319-bbb7-cd7777facdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:52.335075 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83acedd-b9b9-4319-bbb7-cd7777facdb3 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:52.335518 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83acedd-b9b9-4319-bbb7-cd7777facdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:52.336167 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83acedd-b9b9-4319-bbb7-cd7777facdb3 None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:52.338442 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83acedd-b9b9-4319-bbb7-cd7777facdb3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:52.338921 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83acedd-b9b9-4319-bbb7-cd7777facdb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:52.340712 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83acedd-b9b9-4319-bbb7-cd7777facdb3 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:52.351810 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 468/920] 10.222.0.59 () {56 vars in 1122 bytes} [Fri Aug 26 00:35:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:52.353857 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83acedd-b9b9-4319-bbb7-cd7777facdb3 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:52.366168 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1129ef-5fb7-46fd-a0b8-cbd35ea29888 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:52.388305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:52.415621 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7463250-c1ab-4bfb-9200-308f9b3c1bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:52.416386 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7463250-c1ab-4bfb-9200-308f9b3c1bc1 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:52.416869 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7463250-c1ab-4bfb-9200-308f9b3c1bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:52.417238 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7463250-c1ab-4bfb-9200-308f9b3c1bc1 None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:52.418021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7463250-c1ab-4bfb-9200-308f9b3c1bc1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:52.418567 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7463250-c1ab-4bfb-9200-308f9b3c1bc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:52.420224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7463250-c1ab-4bfb-9200-308f9b3c1bc1 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:52.431813 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 453/921] 10.222.0.59 () {56 vars in 1114 bytes} [Fri Aug 26 00:35:52 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:52.433026 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7463250-c1ab-4bfb-9200-308f9b3c1bc1 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:52.453558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:52.513173 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83acedd-b9b9-4319-bbb7-cd7777facdb3 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:52.551107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7463250-c1ab-4bfb-9200-308f9b3c1bc1 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:52.558944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea459b-02ab-4a69-8b9b-b71ba672f81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:52.560116 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea459b-02ab-4a69-8b9b-b71ba672f81f None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:52.560506 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea459b-02ab-4a69-8b9b-b71ba672f81f None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:52.560887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea459b-02ab-4a69-8b9b-b71ba672f81f None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:52.561647 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea459b-02ab-4a69-8b9b-b71ba672f81f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:52.562038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea459b-02ab-4a69-8b9b-b71ba672f81f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:52.564222 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea459b-02ab-4a69-8b9b-b71ba672f81f None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:52.571130 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 469/922] 10.222.0.59 () {56 vars in 1122 bytes} [Fri Aug 26 00:35:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:35:52.576448 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea459b-02ab-4a69-8b9b-b71ba672f81f None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:52.598427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732be4b-d63c-4ae4-ba5c-dc73d42ebe68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:52.599428 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732be4b-d63c-4ae4-ba5c-dc73d42ebe68 None admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:52.599963 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732be4b-d63c-4ae4-ba5c-dc73d42ebe68 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:52.600957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732be4b-d63c-4ae4-ba5c-dc73d42ebe68 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:52.601976 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7732be4b-d63c-4ae4-ba5c-dc73d42ebe68 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:52.604762 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7732be4b-d63c-4ae4-ba5c-dc73d42ebe68 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:52.677996 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:52.688579 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 454/923] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:35:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:35:52.688879 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea459b-02ab-4a69-8b9b-b71ba672f81f None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:52.740383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5345ca4-17ed-47a4-8f02-5d77e222a7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:52.741547 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5345ca4-17ed-47a4-8f02-5d77e222a7b9 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:52.742039 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5345ca4-17ed-47a4-8f02-5d77e222a7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:52.742528 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5345ca4-17ed-47a4-8f02-5d77e222a7b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:52.743472 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5345ca4-17ed-47a4-8f02-5d77e222a7b9 None admin] RBAC: Adding query filter params (interface=public, service_id=110fe8414d8645a09d1f5d94751b0965, region_id=RegionOne) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:52.744007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5345ca4-17ed-47a4-8f02-5d77e222a7b9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:52.745961 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5345ca4-17ed-47a4-8f02-5d77e222a7b9 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:52.760659 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 470/924] 10.222.0.59 () {56 vars in 1251 bytes} [Fri Aug 26 00:35:52 2022] GET /identity/v3/endpoints?service_id=110fe8414d8645a09d1f5d94751b0965&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:35:52.763106 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732be4b-d63c-4ae4-ba5c-dc73d42ebe68 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:52.840896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:52.874398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732be4b-d63c-4ae4-ba5c-dc73d42ebe68 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:52.910537 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9978fb2a-2cbb-4e9b-aba0-5bd90d7301a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:52.911160 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978fb2a-2cbb-4e9b-aba0-5bd90d7301a6 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:52.911442 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978fb2a-2cbb-4e9b-aba0-5bd90d7301a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:52.911719 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978fb2a-2cbb-4e9b-aba0-5bd90d7301a6 None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:52.912329 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9978fb2a-2cbb-4e9b-aba0-5bd90d7301a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:52.913728 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9978fb2a-2cbb-4e9b-aba0-5bd90d7301a6 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:52.924600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 455/925] 10.222.0.59 () {56 vars in 1087 bytes} [Fri Aug 26 00:35:52 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:35:53.003785 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fd73c-4174-4fae-a91a-f18fa2d0a077 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:53.004100 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fd73c-4174-4fae-a91a-f18fa2d0a077 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:53.004292 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fd73c-4174-4fae-a91a-f18fa2d0a077 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:53.006015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 471/926] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:53.011692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb30bde-d09f-42b0-845b-eee9e5d5c3e3 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:53.012209 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb30bde-d09f-42b0-845b-eee9e5d5c3e3 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:53.012683 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb30bde-d09f-42b0-845b-eee9e5d5c3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:53.032933 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-2eb30bde-d09f-42b0-845b-eee9e5d5c3e3 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:53.034443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-2eb30bde-d09f-42b0-845b-eee9e5d5c3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:53.109448 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 456/927] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:35:53.384919 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53fd73c-4174-4fae-a91a-f18fa2d0a077 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:53.503949 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53fd73c-4174-4fae-a91a-f18fa2d0a077 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:53.539545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ae8cc-bc89-4de0-8a5c-5e8661a3bd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:53.540461 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ae8cc-bc89-4de0-8a5c-5e8661a3bd1e None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:53.540927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ae8cc-bc89-4de0-8a5c-5e8661a3bd1e None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:53.541348 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ae8cc-bc89-4de0-8a5c-5e8661a3bd1e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:53.542091 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ae8cc-bc89-4de0-8a5c-5e8661a3bd1e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:53.543793 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ae8cc-bc89-4de0-8a5c-5e8661a3bd1e None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:53.552899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-158ae8cc-bc89-4de0-8a5c-5e8661a3bd1e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 00:35:53.552899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:53.552899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 00:35:53.552899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 00:35:53.552899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 00:35:53.552899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 00:35:53.552899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 00:35:53.552899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:53.552899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 00:35:53.552899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:53.552899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:35:53.552899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:53.552899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:53.552899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:53.552899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:53.552899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:53.554085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:53.554085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:53.554085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:53.554085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:53.554085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:53.554085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:53.554085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:53.554085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 00:35:53.554085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 00:35:53.554085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 00:35:53.554085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 00:35:53.554085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:53.554085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:53.554085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:53.554085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:53.554085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:53.554085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:53.555180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:53.555180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:53.555180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:53.555180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:53.555180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:53.555180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:53.555180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:53.555180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:53.555180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:53.555180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:53.555180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 00:35:53.555180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:53.555180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 00:35:53.555180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:53.557605 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 472/928] 10.222.0.59 () {56 vars in 1132 bytes} [Fri Aug 26 00:35:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:53.563976 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb30bde-d09f-42b0-845b-eee9e5d5c3e3 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:53.631572 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:53.683433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb30bde-d09f-42b0-845b-eee9e5d5c3e3 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:53.721203 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ed3bb-e28f-4be8-875f-d3ce9c85f437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:53.722100 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ed3bb-e28f-4be8-875f-d3ce9c85f437 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:53.722529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ed3bb-e28f-4be8-875f-d3ce9c85f437 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:53.722961 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ed3bb-e28f-4be8-875f-d3ce9c85f437 None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:53.723764 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ed3bb-e28f-4be8-875f-d3ce9c85f437 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:53.724214 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ed3bb-e28f-4be8-875f-d3ce9c85f437 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:53.725767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ed3bb-e28f-4be8-875f-d3ce9c85f437 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:53.731441 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 457/929] 10.222.0.59 () {56 vars in 1126 bytes} [Fri Aug 26 00:35:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:53.737410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ae8cc-bc89-4de0-8a5c-5e8661a3bd1e None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:53.754890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:53.842992 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ae8cc-bc89-4de0-8a5c-5e8661a3bd1e None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:53.885006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb9cc5-45d0-4ba9-a5b6-b712d4ec2173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:53.885622 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb9cc5-45d0-4ba9-a5b6-b712d4ec2173 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:53.885905 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb9cc5-45d0-4ba9-a5b6-b712d4ec2173 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:53.886182 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb9cc5-45d0-4ba9-a5b6-b712d4ec2173 None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:53.886819 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb9cc5-45d0-4ba9-a5b6-b712d4ec2173 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:53.887211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb9cc5-45d0-4ba9-a5b6-b712d4ec2173 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:53.888613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb9cc5-45d0-4ba9-a5b6-b712d4ec2173 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:53.894540 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 473/930] 10.222.0.59 () {56 vars in 1126 bytes} [Fri Aug 26 00:35:53 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:53.900492 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ed3bb-e28f-4be8-875f-d3ce9c85f437 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:54.006399 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ed3bb-e28f-4be8-875f-d3ce9c85f437 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:54.011437 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2dc44c-4bc5-42f5-a697-f4724ece6c9e None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:54.011724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2dc44c-4bc5-42f5-a697-f4724ece6c9e None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:54.012000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2dc44c-4bc5-42f5-a697-f4724ece6c9e None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:54.013616 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 474/931] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:54.018684 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0deba4-b04b-401c-aaff-4968fdb21a74 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:54.021301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0deba4-b04b-401c-aaff-4968fdb21a74 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:54.021301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0deba4-b04b-401c-aaff-4968fdb21a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:54.038942 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-be0deba4-b04b-401c-aaff-4968fdb21a74 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:54.040409 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-be0deba4-b04b-401c-aaff-4968fdb21a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:54.044300 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:54.049234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708e0bf-dfae-4ac7-9625-a49f9f3208f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:54.050133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708e0bf-dfae-4ac7-9625-a49f9f3208f0 None admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:54.050591 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708e0bf-dfae-4ac7-9625-a49f9f3208f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:54.051112 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708e0bf-dfae-4ac7-9625-a49f9f3208f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:54.051980 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5708e0bf-dfae-4ac7-9625-a49f9f3208f0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:54.053713 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5708e0bf-dfae-4ac7-9625-a49f9f3208f0 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:54.112508 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 475/932] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:35:54.152329 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 458/933] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:35:53 2022] POST /identity/v3/endpoints => generated 347 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:35:54.399073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0deba4-b04b-401c-aaff-4968fdb21a74 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:54.410593 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:54.518558 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0deba4-b04b-401c-aaff-4968fdb21a74 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:54.560504 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee2120-7f0f-4482-a124-04dabb97b2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:54.561181 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee2120-7f0f-4482-a124-04dabb97b2e7 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:54.561459 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee2120-7f0f-4482-a124-04dabb97b2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:54.561729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee2120-7f0f-4482-a124-04dabb97b2e7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:54.572704 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee2120-7f0f-4482-a124-04dabb97b2e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:54.577462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-42ee2120-7f0f-4482-a124-04dabb97b2e7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 00:35:54.577462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:54.577462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:54.577462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:54.577462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:54.577462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:54.577462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:54.577462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:54.577462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:54.577462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:54.577462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:54.577462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:54.577462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:54.577462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:54.577462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:54.577462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:54.578451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:54.578451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:54.578451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:54.578451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:54.578451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:54.578451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:54.578451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:54.578451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:54.578451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:54.578451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:54.578451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:54.578451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:54.578451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:54.578451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:54.578451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:54.578451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:54.579323 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:54.579323 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:54.579323 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:54.579323 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:54.579323 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:54.579323 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:54.579323 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:54.579323 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:54.579323 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:54.579323 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:54.579323 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 00:35:54.579323 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:54.579323 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 476/934] 10.222.0.59 () {56 vars in 1120 bytes} [Fri Aug 26 00:35:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:54.586505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708e0bf-dfae-4ac7-9625-a49f9f3208f0 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:54.699293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708e0bf-dfae-4ac7-9625-a49f9f3208f0 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:54.734110 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda18e7c-14fa-4049-b655-0a3489b9d9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:54.734724 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda18e7c-14fa-4049-b655-0a3489b9d9fc None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:54.735000 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda18e7c-14fa-4049-b655-0a3489b9d9fc None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:54.735266 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda18e7c-14fa-4049-b655-0a3489b9d9fc None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:54.735931 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda18e7c-14fa-4049-b655-0a3489b9d9fc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:54.736229 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda18e7c-14fa-4049-b655-0a3489b9d9fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:54.737629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda18e7c-14fa-4049-b655-0a3489b9d9fc None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:54.746505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 459/935] 10.222.0.59 () {56 vars in 1115 bytes} [Fri Aug 26 00:35:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:54.752075 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee2120-7f0f-4482-a124-04dabb97b2e7 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:54.784769 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597962e7-0f1d-4cff-8571-0f4be7f95dcd None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:54.785108 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597962e7-0f1d-4cff-8571-0f4be7f95dcd None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:54.785598 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597962e7-0f1d-4cff-8571-0f4be7f95dcd None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:54.787621 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 460/936] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:54.792701 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d0cb6-a922-485a-9048-435aeaf04356 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:54.792985 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d0cb6-a922-485a-9048-435aeaf04356 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:54.793316 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d0cb6-a922-485a-9048-435aeaf04356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:54.814865 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-213d0cb6-a922-485a-9048-435aeaf04356 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:54.816056 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-213d0cb6-a922-485a-9048-435aeaf04356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:54.826346 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:54.871765 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee2120-7f0f-4482-a124-04dabb97b2e7 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:54.896879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:54.900529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 461/937] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:35:54.915542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563e4ed-1c47-497b-913a-6e223379b29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:54.916512 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563e4ed-1c47-497b-913a-6e223379b29f None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:54.917016 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563e4ed-1c47-497b-913a-6e223379b29f None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:54.917487 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563e4ed-1c47-497b-913a-6e223379b29f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:54.928793 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563e4ed-1c47-497b-913a-6e223379b29f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:54.930495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563e4ed-1c47-497b-913a-6e223379b29f None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:54.945061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-2563e4ed-1c47-497b-913a-6e223379b29f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:54.945061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:54.945061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:54.945061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:54.945061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:54.945061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:54.945061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:54.945061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:54.945061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:54.945061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:54.945061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:54.945061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:54.945061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:54.945061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:54.945061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:54.945061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:54.946047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:54.946047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:54.946047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:54.946047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:54.946047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:54.946047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:54.946047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:54.946047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:54.946047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:54.946047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:54.946047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:54.946047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:54.946047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:54.946047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:54.946047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:54.946047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:54.946047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:54.946960 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:54.946960 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:54.946960 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:54.949081 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 477/938] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:54.958707 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d0cb6-a922-485a-9048-435aeaf04356 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:55.050008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e87ab2-3c03-4cd2-aa11-072a3d7e8a87 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:55.050343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e87ab2-3c03-4cd2-aa11-072a3d7e8a87 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:55.050558 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e87ab2-3c03-4cd2-aa11-072a3d7e8a87 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:55.052017 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 478/939] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:35:55.057057 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd77af1-fbfd-4e1e-a452-52c9ebcd97dd None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:55.057346 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd77af1-fbfd-4e1e-a452-52c9ebcd97dd None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:55.057619 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd77af1-fbfd-4e1e-a452-52c9ebcd97dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:55.069388 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d0cb6-a922-485a-9048-435aeaf04356 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:55.078479 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-abd77af1-fbfd-4e1e-a452-52c9ebcd97dd None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:55.080146 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-abd77af1-fbfd-4e1e-a452-52c9ebcd97dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:55.111502 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ea754-b675-4614-9931-320ae94c040f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:55.112615 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ea754-b675-4614-9931-320ae94c040f None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:55.113238 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ea754-b675-4614-9931-320ae94c040f None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:55.113764 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ea754-b675-4614-9931-320ae94c040f None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:55.114615 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ea754-b675-4614-9931-320ae94c040f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:55.115110 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ea754-b675-4614-9931-320ae94c040f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:55.116641 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ea754-b675-4614-9931-320ae94c040f None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:55.130128 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 462/940] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:55.140667 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ea754-b675-4614-9931-320ae94c040f None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:55.157728 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 479/941] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:35:55.182508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:55.186716 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd77af1-fbfd-4e1e-a452-52c9ebcd97dd None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:55.256603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:55.274332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ea754-b675-4614-9931-320ae94c040f None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:55.309117 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c66754-b0b7-498b-99eb-e52c7319310a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:55.309755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c66754-b0b7-498b-99eb-e52c7319310a None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:55.310044 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c66754-b0b7-498b-99eb-e52c7319310a None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:55.310497 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c66754-b0b7-498b-99eb-e52c7319310a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:55.318031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c66754-b0b7-498b-99eb-e52c7319310a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:55.319484 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c66754-b0b7-498b-99eb-e52c7319310a None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:55.321013 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd77af1-fbfd-4e1e-a452-52c9ebcd97dd None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:55.331887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-e4c66754-b0b7-498b-99eb-e52c7319310a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 00:35:55.331887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:55.331887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:55.331887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:55.331887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:55.331887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:55.331887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:55.331887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:55.331887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:55.331887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:55.331887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:55.331887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:55.331887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:35:55.331887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:35:55.331887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:35:55.331887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:35:55.332966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:55.332966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:55.332966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:55.332966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:55.332966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:55.332966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:55.332966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:55.332966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:55.332966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:55.332966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:55.332966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:55.332966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:55.332966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:55.332966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:55.332966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:55.332966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:55.332966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:55.332966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:55.333971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:55.333971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:55.333971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:35:55.333971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:35:55.333971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:35:55.333971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:35:55.333971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:35:55.333971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:35:55.333971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 00:35:55.333971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:55.333971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 463/942] 10.222.0.59 () {56 vars in 1117 bytes} [Fri Aug 26 00:35:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:55.341424 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c66754-b0b7-498b-99eb-e52c7319310a None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:55.368091 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a12d9-0dbc-4556-882a-b6d255b2388a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:55.368800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a12d9-0dbc-4556-882a-b6d255b2388a None admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:55.369079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a12d9-0dbc-4556-882a-b6d255b2388a None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:55.369404 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a12d9-0dbc-4556-882a-b6d255b2388a None admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:55.370070 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a12d9-0dbc-4556-882a-b6d255b2388a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:55.371612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a12d9-0dbc-4556-882a-b6d255b2388a None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:55.375643 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: WARNING keystone.server.flask.common [None req-bb0a12d9-0dbc-4556-882a-b6d255b2388a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 26 00:35:55.425718 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.sql.core [None req-bb0a12d9-0dbc-4556-882a-b6d255b2388a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 26 00:35:55.425718 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 26 00:35:55.425718 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [parameters: {'id': '9b99d952be56466abb3eb2324c716d0e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 26 00:35:55.425718 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88254) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 26 00:35:55.435509 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-bb0a12d9-0dbc-4556-882a-b6d255b2388a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9b99d952be56466abb3eb2324c716d0e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9b99d952be56466abb3eb2324c716d0e. Aug 26 00:35:55.435509 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:55.435509 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 00:35:55.435509 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 00:35:55.435509 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 00:35:55.435509 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 00:35:55.435509 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 00:35:55.435509 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 00:35:55.435509 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 00:35:55.435509 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application conn.query(q) Aug 26 00:35:55.435509 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 00:35:55.435509 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 00:35:55.435509 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 00:35:55.436510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application result.read() Aug 26 00:35:55.436510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 00:35:55.436510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 00:35:55.436510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 00:35:55.436510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 00:35:55.436510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 00:35:55.436510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 00:35:55.436510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 00:35:55.436510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 00:35:55.436510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 26 00:35:55.436510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:55.436510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 26 00:35:55.436510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:55.436510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:55.436510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 26 00:35:55.436510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 26 00:35:55.436510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 26 00:35:55.436510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 26 00:35:55.437483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 00:35:55.437483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application next(self.gen) Aug 26 00:35:55.437483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 26 00:35:55.437483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application yield resource Aug 26 00:35:55.437483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 00:35:55.437483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application next(self.gen) Aug 26 00:35:55.437483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 26 00:35:55.437483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self.session.rollback() Aug 26 00:35:55.437483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 26 00:35:55.437483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self.force_reraise() Aug 26 00:35:55.437483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 26 00:35:55.437483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise self.value Aug 26 00:35:55.437483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 26 00:35:55.437483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 26 00:35:55.437483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 26 00:35:55.437483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application session.commit() Aug 26 00:35:55.437483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 26 00:35:55.438397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 26 00:35:55.438397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 26 00:35:55.438397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._prepare_impl() Aug 26 00:35:55.438397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 26 00:35:55.438397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self.session.flush() Aug 26 00:35:55.438397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 26 00:35:55.438397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._flush(objects) Aug 26 00:35:55.438397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 26 00:35:55.438397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 26 00:35:55.438397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 26 00:35:55.438397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application compat.raise_( Aug 26 00:35:55.438397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 00:35:55.438397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception Aug 26 00:35:55.438397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 26 00:35:55.438397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application flush_context.execute() Aug 26 00:35:55.438397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 26 00:35:55.438397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rec.execute(self) Aug 26 00:35:55.439345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 26 00:35:55.439345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 26 00:35:55.439345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 26 00:35:55.439345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 26 00:35:55.439345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 26 00:35:55.439345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 26 00:35:55.439345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 26 00:35:55.439345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 26 00:35:55.439345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 26 00:35:55.439345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 26 00:35:55.439345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 26 00:35:55.439345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 26 00:35:55.439345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 26 00:35:55.439345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 26 00:35:55.439345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 26 00:35:55.440164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 26 00:35:55.440164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 00:35:55.440164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception Aug 26 00:35:55.440164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 00:35:55.440164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 00:35:55.440164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 00:35:55.440164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 00:35:55.440164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 00:35:55.440164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 00:35:55.440164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 00:35:55.440164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application conn.query(q) Aug 26 00:35:55.440164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 00:35:55.440164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 00:35:55.440164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 00:35:55.440164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application result.read() Aug 26 00:35:55.440164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 00:35:55.440164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 00:35:55.441073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 00:35:55.441073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 00:35:55.441073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 00:35:55.441073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 00:35:55.441073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 00:35:55.441073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 00:35:55.441073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 26 00:35:55.441073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 26 00:35:55.441073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [parameters: {'id': '9b99d952be56466abb3eb2324c716d0e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 26 00:35:55.441073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 26 00:35:55.441073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:55.441073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:35:55.441073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:55.441073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:55.441073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 26 00:35:55.441871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 26 00:35:55.441871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 26 00:35:55.441871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 26 00:35:55.441871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 26 00:35:55.441871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:55.441871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:35:55.441871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:55.441871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:55.441871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:55.441871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:55.441871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:55.441871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:55.441871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:55.441871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:55.441871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:55.441871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:55.441871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:55.443215 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:55.443215 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 26 00:35:55.443215 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 26 00:35:55.443215 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:55.443215 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:55.443215 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 26 00:35:55.443215 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 26 00:35:55.443215 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9b99d952be56466abb3eb2324c716d0e. Aug 26 00:35:55.443215 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:55.443215 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 480/943] 10.222.0.59 () {60 vars in 1140 bytes} [Fri Aug 26 00:35:55 2022] POST /identity/v3/projects => generated 262 bytes in 252 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:55.443215 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a12d9-0dbc-4556-882a-b6d255b2388a None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:55.461859 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:55.469651 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c66754-b0b7-498b-99eb-e52c7319310a None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:55.502328 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12803df8-3fb3-42d1-976c-b4b9f92cef4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:55.502924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12803df8-3fb3-42d1-976c-b4b9f92cef4e None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:55.503204 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12803df8-3fb3-42d1-976c-b4b9f92cef4e None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:55.503489 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12803df8-3fb3-42d1-976c-b4b9f92cef4e None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:55.504187 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12803df8-3fb3-42d1-976c-b4b9f92cef4e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:55.504524 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12803df8-3fb3-42d1-976c-b4b9f92cef4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:55.505927 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12803df8-3fb3-42d1-976c-b4b9f92cef4e None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:55.523814 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 464/944] 10.222.0.59 () {56 vars in 1149 bytes} [Fri Aug 26 00:35:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:55.524747 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12803df8-3fb3-42d1-976c-b4b9f92cef4e None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:55.549251 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a12d9-0dbc-4556-882a-b6d255b2388a None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:55.550994 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:55.587332 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2188901-71fa-4a67-b067-29c29232faaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:55.588070 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2188901-71fa-4a67-b067-29c29232faaa None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:55.588357 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2188901-71fa-4a67-b067-29c29232faaa None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:55.588824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2188901-71fa-4a67-b067-29c29232faaa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:55.589523 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2188901-71fa-4a67-b067-29c29232faaa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:55.591111 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2188901-71fa-4a67-b067-29c29232faaa None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:55.600587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-f2188901-71fa-4a67-b067-29c29232faaa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 00:35:55.600587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:55.600587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 00:35:55.600587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 00:35:55.600587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 00:35:55.600587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 00:35:55.600587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 00:35:55.600587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:55.600587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 00:35:55.600587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:55.600587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:35:55.600587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:55.600587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:55.600587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:55.600587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:55.600587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:55.600587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:55.601988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:55.601988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:55.601988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:55.601988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:55.601988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:55.601988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:55.601988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 00:35:55.601988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 00:35:55.601988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 00:35:55.601988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 00:35:55.601988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:55.601988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:55.601988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:55.601988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:55.601988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:55.601988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:55.603298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:55.603298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:55.603298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:55.603298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:55.603298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:55.603298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:55.603298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:55.603298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:55.603298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:55.603298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:55.603298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 00:35:55.603298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:55.603298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 00:35:55.603298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:55.603298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 481/945] 10.222.0.59 () {56 vars in 1126 bytes} [Fri Aug 26 00:35:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:55.604309 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2188901-71fa-4a67-b067-29c29232faaa None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:55.631243 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12803df8-3fb3-42d1-976c-b4b9f92cef4e None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:55.671034 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6928b-9b5c-49f2-8a24-dc4535b333d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:55.671678 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6928b-9b5c-49f2-8a24-dc4535b333d5 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:55.671964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6928b-9b5c-49f2-8a24-dc4535b333d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:55.672241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6928b-9b5c-49f2-8a24-dc4535b333d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:55.679142 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c6928b-9b5c-49f2-8a24-dc4535b333d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:55.680882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c6928b-9b5c-49f2-8a24-dc4535b333d5 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:55.694692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-87c6928b-9b5c-49f2-8a24-dc4535b333d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:55.694692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:55.694692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:55.694692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:55.694692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:55.694692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:55.694692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:55.694692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:55.694692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:55.694692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:55.694692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:55.694692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:55.694692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:35:55.694692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:35:55.694692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:35:55.695717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:35:55.695717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:55.695717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:55.695717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:55.695717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:55.695717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:55.695717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:55.695717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:55.695717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:55.695717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:55.695717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:55.695717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:55.695717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:55.695717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:55.695717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:55.695717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:55.695717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:55.696700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:35:55.696700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:35:55.696700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:35:55.696700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:35:55.696700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:35:55.696700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:35:55.696700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:55.696700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:55.696700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 465/946] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:55.697733 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6928b-9b5c-49f2-8a24-dc4535b333d5 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:55.717223 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:55.720125 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2188901-71fa-4a67-b067-29c29232faaa None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:55.755324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbcf325-3d11-4e38-b94d-6f98043f6999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:55.755972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcf325-3d11-4e38-b94d-6f98043f6999 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:55.756250 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcf325-3d11-4e38-b94d-6f98043f6999 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:55.756558 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcf325-3d11-4e38-b94d-6f98043f6999 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:55.767849 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbcf325-3d11-4e38-b94d-6f98043f6999 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:55.769382 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbcf325-3d11-4e38-b94d-6f98043f6999 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:55.784012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-abbcf325-3d11-4e38-b94d-6f98043f6999 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:55.784012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:55.784012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:55.784012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:55.784012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:55.784012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:55.784012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:55.784012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:55.784012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:55.784012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:55.784012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:55.784012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:55.784012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:55.784012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:55.784012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:55.784012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:55.785006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:55.785006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:55.785006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:55.785006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:55.785006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:55.785006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:55.785006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:55.785006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:55.785006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:55.785006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:55.785006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:55.785006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:55.785006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:55.785006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:55.785006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:55.785006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:55.785006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:55.785946 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:55.785946 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:55.785946 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:55.789992 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 482/947] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:55.790669 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbcf325-3d11-4e38-b94d-6f98043f6999 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:55.808460 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6928b-9b5c-49f2-8a24-dc4535b333d5 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:55.812181 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:55.846431 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7341868-d4fb-41ba-98d5-3413ace75edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:55.847052 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7341868-d4fb-41ba-98d5-3413ace75edb None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:55.847327 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7341868-d4fb-41ba-98d5-3413ace75edb None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:55.847605 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7341868-d4fb-41ba-98d5-3413ace75edb None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:55.848243 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7341868-d4fb-41ba-98d5-3413ace75edb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:55.848574 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7341868-d4fb-41ba-98d5-3413ace75edb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:55.849939 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7341868-d4fb-41ba-98d5-3413ace75edb None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:55.855040 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 466/948] 10.222.0.59 () {56 vars in 1122 bytes} [Fri Aug 26 00:35:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:55.855643 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7341868-d4fb-41ba-98d5-3413ace75edb None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:55.907356 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbcf325-3d11-4e38-b94d-6f98043f6999 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:55.951758 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a9c7e-46f6-486a-aaea-6744e31d4e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:55.952277 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:55.952417 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a9c7e-46f6-486a-aaea-6744e31d4e1f None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:55.952680 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a9c7e-46f6-486a-aaea-6744e31d4e1f None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:55.953078 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a9c7e-46f6-486a-aaea-6744e31d4e1f None admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:55.953851 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a9c7e-46f6-486a-aaea-6744e31d4e1f None admin] RBAC: Adding query filter params (name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:55.954751 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a9c7e-46f6-486a-aaea-6744e31d4e1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:55.956242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a9c7e-46f6-486a-aaea-6744e31d4e1f None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:55.974431 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7341868-d4fb-41ba-98d5-3413ace75edb None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:55.979168 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 483/949] 10.222.0.59 () {56 vars in 1112 bytes} [Fri Aug 26 00:35:55 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:55.979861 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a9c7e-46f6-486a-aaea-6744e31d4e1f None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:56.021292 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d146c-a7ef-48a4-a3c0-2c5a5b824220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:56.021995 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d146c-a7ef-48a4-a3c0-2c5a5b824220 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:56.022373 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d146c-a7ef-48a4-a3c0-2c5a5b824220 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:56.023026 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d146c-a7ef-48a4-a3c0-2c5a5b824220 None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:56.023721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d146c-a7ef-48a4-a3c0-2c5a5b824220 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:56.024089 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d146c-a7ef-48a4-a3c0-2c5a5b824220 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:56.025539 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d146c-a7ef-48a4-a3c0-2c5a5b824220 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:56.051029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 467/950] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:56.051696 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d146c-a7ef-48a4-a3c0-2c5a5b824220 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:56.101322 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:56.125224 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a9c7e-46f6-486a-aaea-6744e31d4e1f None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:56.169690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb11b1-83f7-43ef-a237-ce8730c2482d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:56.170610 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb11b1-83f7-43ef-a237-ce8730c2482d None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:56.171030 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb11b1-83f7-43ef-a237-ce8730c2482d None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:56.171430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb11b1-83f7-43ef-a237-ce8730c2482d None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:56.172197 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb11b1-83f7-43ef-a237-ce8730c2482d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:56.172805 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb11b1-83f7-43ef-a237-ce8730c2482d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:56.174332 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb11b1-83f7-43ef-a237-ce8730c2482d None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:56.183783 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d146c-a7ef-48a4-a3c0-2c5a5b824220 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:56.184021 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 484/951] 10.222.0.59 () {56 vars in 1122 bytes} [Fri Aug 26 00:35:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:56.185551 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb11b1-83f7-43ef-a237-ce8730c2482d None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:56.231649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e98c24-99e1-4034-b3e3-c26743d7c869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:56.232305 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e98c24-99e1-4034-b3e3-c26743d7c869 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:56.232616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e98c24-99e1-4034-b3e3-c26743d7c869 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:56.233421 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e98c24-99e1-4034-b3e3-c26743d7c869 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:56.244653 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e98c24-99e1-4034-b3e3-c26743d7c869 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:56.244653 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e98c24-99e1-4034-b3e3-c26743d7c869 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:56.259763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-f2e98c24-99e1-4034-b3e3-c26743d7c869 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:56.259763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:56.259763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:56.259763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:56.259763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:56.259763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:56.259763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:56.259763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:56.259763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:56.259763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:56.259763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:56.259763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:56.259763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:35:56.259763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:35:56.259763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:35:56.260984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:35:56.260984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:56.260984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:56.260984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:56.260984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:56.260984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:56.260984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:56.260984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:56.260984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:56.260984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:56.260984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:56.260984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:56.260984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:56.260984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:56.260984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:56.260984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:56.260984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:56.262167 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:35:56.262167 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:35:56.262167 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:35:56.262167 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:35:56.262167 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:35:56.262167 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:35:56.262167 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:35:56.262167 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:56.262167 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:56.264685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 468/952] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:56.266853 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e98c24-99e1-4034-b3e3-c26743d7c869 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:56.304937 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb11b1-83f7-43ef-a237-ce8730c2482d None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:56.340711 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dcac62-3ef9-4232-8c6c-83a907820ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:56.341474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcac62-3ef9-4232-8c6c-83a907820ff2 None admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:56.341849 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcac62-3ef9-4232-8c6c-83a907820ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:56.342251 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcac62-3ef9-4232-8c6c-83a907820ff2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:56.342997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dcac62-3ef9-4232-8c6c-83a907820ff2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:56.344446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dcac62-3ef9-4232-8c6c-83a907820ff2 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:56.381978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e98c24-99e1-4034-b3e3-c26743d7c869 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:56.408861 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:56.424500 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 485/953] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:35:56 2022] POST /identity/v3/endpoints => generated 349 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:35:56.424500 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1bb9d1-6b3e-4c16-a7f4-0c145190db9c None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:56.424500 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1bb9d1-6b3e-4c16-a7f4-0c145190db9c None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:56.424500 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1bb9d1-6b3e-4c16-a7f4-0c145190db9c None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:56.424500 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 486/954] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:56.425563 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef5073-56ca-48db-863f-ed6d29532471 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:56.425850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef5073-56ca-48db-863f-ed6d29532471 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:56.426392 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef5073-56ca-48db-863f-ed6d29532471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:56.439273 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf650bb-49de-4176-90b6-873d7b33ac13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:56.440198 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf650bb-49de-4176-90b6-873d7b33ac13 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:56.440594 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf650bb-49de-4176-90b6-873d7b33ac13 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:56.441013 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf650bb-49de-4176-90b6-873d7b33ac13 None admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:56.441908 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf650bb-49de-4176-90b6-873d7b33ac13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:56.442317 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf650bb-49de-4176-90b6-873d7b33ac13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:56.445161 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf650bb-49de-4176-90b6-873d7b33ac13 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:56.449710 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-99ef5073-56ca-48db-863f-ed6d29532471 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:56.451489 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-99ef5073-56ca-48db-863f-ed6d29532471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:56.463320 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 469/955] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:35:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:35:56.469266 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf650bb-49de-4176-90b6-873d7b33ac13 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:56.537720 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 487/956] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:35:56.568979 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:56.608503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf650bb-49de-4176-90b6-873d7b33ac13 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:56.643410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2693a-20bc-4290-a680-409b8c53e822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:56.644160 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2693a-20bc-4290-a680-409b8c53e822 None admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:56.644353 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2693a-20bc-4290-a680-409b8c53e822 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:56.644663 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2693a-20bc-4290-a680-409b8c53e822 None admin] PATH_INFO: `/v3/projects/fde7ac3ed472433ab3bba56e5e091510/users/bbd9bbb083ab4f5e84e135ec5103cdae/roles/91940e122e2f46e3818c5f11f4fca9e9` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:56.646852 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2693a-20bc-4290-a680-409b8c53e822 None admin] RBAC: Authorizing `identity:create_grant(project_id=fde7ac3ed472433ab3bba56e5e091510, user_id=bbd9bbb083ab4f5e84e135ec5103cdae, role_id=91940e122e2f46e3818c5f11f4fca9e9)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:56.648325 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2693a-20bc-4290-a680-409b8c53e822 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:56.695657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 470/957] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Aug 26 00:35:56 2022] PUT /identity/v3/projects/fde7ac3ed472433ab3bba56e5e091510/users/bbd9bbb083ab4f5e84e135ec5103cdae/roles/91940e122e2f46e3818c5f11f4fca9e9 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:35:56.818865 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef5073-56ca-48db-863f-ed6d29532471 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:56.934446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef5073-56ca-48db-863f-ed6d29532471 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:56.962308 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:56.971260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa65d34-95bb-464a-8a37-693bb901d904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:56.971909 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa65d34-95bb-464a-8a37-693bb901d904 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:56.972189 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa65d34-95bb-464a-8a37-693bb901d904 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:56.972466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa65d34-95bb-464a-8a37-693bb901d904 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:56.973084 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa65d34-95bb-464a-8a37-693bb901d904 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:56.974585 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa65d34-95bb-464a-8a37-693bb901d904 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:56.985296 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-baa65d34-95bb-464a-8a37-693bb901d904 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 00:35:56.985296 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:56.985296 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 00:35:56.985296 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 00:35:56.985296 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 00:35:56.985296 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 00:35:56.985296 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 00:35:56.985296 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:56.985296 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 00:35:56.985296 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:56.985296 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:35:56.985296 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:56.985296 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:56.985296 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:56.985296 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:56.985296 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:56.985296 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:56.986568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:56.986568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:56.986568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:56.986568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:56.986568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:56.986568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:56.986568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 00:35:56.986568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 00:35:56.986568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 00:35:56.986568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 00:35:56.986568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:56.986568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:56.986568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:56.986568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:56.986568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:56.986568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:56.987644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:56.987644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:56.987644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:56.987644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:56.987644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:56.987644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:56.987644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:56.987644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:56.987644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:56.987644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:56.987644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 00:35:56.987644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:56.987644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 00:35:56.987644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:56.990358 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 488/958] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:35:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:56.991611 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2693a-20bc-4290-a680-409b8c53e822 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:57.099347 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2693a-20bc-4290-a680-409b8c53e822 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:57.113674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:57.133850 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8069bc-e8f3-4bfe-9f3d-1d7cdef2b029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:57.134606 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8069bc-e8f3-4bfe-9f3d-1d7cdef2b029 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:57.134985 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8069bc-e8f3-4bfe-9f3d-1d7cdef2b029 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:57.135374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8069bc-e8f3-4bfe-9f3d-1d7cdef2b029 None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:57.136105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8069bc-e8f3-4bfe-9f3d-1d7cdef2b029 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:57.136528 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8069bc-e8f3-4bfe-9f3d-1d7cdef2b029 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:57.138011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8069bc-e8f3-4bfe-9f3d-1d7cdef2b029 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:57.144118 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 471/959] 10.222.0.59 () {56 vars in 1112 bytes} [Fri Aug 26 00:35:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:57.149772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa65d34-95bb-464a-8a37-693bb901d904 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:57.261409 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa65d34-95bb-464a-8a37-693bb901d904 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:57.298843 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6675a-fe27-40db-9025-75533159faec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:57.299457 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6675a-fe27-40db-9025-75533159faec None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:57.299736 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6675a-fe27-40db-9025-75533159faec None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:57.300008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6675a-fe27-40db-9025-75533159faec None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:57.300671 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6675a-fe27-40db-9025-75533159faec None admin] RBAC: Adding query filter params (type=compute) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:57.300983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6675a-fe27-40db-9025-75533159faec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:57.302336 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6675a-fe27-40db-9025-75533159faec None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:57.312731 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 489/960] 10.222.0.59 () {56 vars in 1112 bytes} [Fri Aug 26 00:35:57 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:58.129046 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5921c-0d08-498c-b4f1-4c436b3a7842 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:58.129581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5921c-0d08-498c-b4f1-4c436b3a7842 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:58.129888 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5921c-0d08-498c-b4f1-4c436b3a7842 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:58.131601 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 472/961] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:58.136965 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b647029-4ee0-43b8-a227-e9e5501cfc22 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:58.137375 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b647029-4ee0-43b8-a227-e9e5501cfc22 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:58.137694 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b647029-4ee0-43b8-a227-e9e5501cfc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:58.157081 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-0b647029-4ee0-43b8-a227-e9e5501cfc22 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:58.158273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-0b647029-4ee0-43b8-a227-e9e5501cfc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:58.212320 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:58.230453 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 490/962] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:35:58.445505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6928e-b465-400b-95fe-6329f31ef2a2 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:58.446120 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6928e-b465-400b-95fe-6329f31ef2a2 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:58.446617 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6928e-b465-400b-95fe-6329f31ef2a2 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:58.448397 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 473/963] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:58.454424 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd138dfc-cef7-48e5-800c-fac9954ae25b None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:58.454945 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd138dfc-cef7-48e5-800c-fac9954ae25b None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:58.455419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd138dfc-cef7-48e5-800c-fac9954ae25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:58.474179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-cd138dfc-cef7-48e5-800c-fac9954ae25b None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:58.475624 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-cd138dfc-cef7-48e5-800c-fac9954ae25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:58.531246 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6928e-b465-400b-95fe-6329f31ef2a2 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:58.549616 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 491/964] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:35:58.636454 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6928e-b465-400b-95fe-6329f31ef2a2 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:58.666776 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2055a78-5ae9-44f9-a40c-2c0d5e8ad4fa None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:58.667087 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2055a78-5ae9-44f9-a40c-2c0d5e8ad4fa None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:58.667332 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2055a78-5ae9-44f9-a40c-2c0d5e8ad4fa None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:58.668960 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 492/965] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:58.674256 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30442b61-0882-4f8a-87c8-0780b1c43770 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:58.674538 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30442b61-0882-4f8a-87c8-0780b1c43770 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:58.674812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30442b61-0882-4f8a-87c8-0780b1c43770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:58.687917 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7157ee-109f-4dca-a539-1afb4c0dbf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:58.688502 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7157ee-109f-4dca-a539-1afb4c0dbf7b None admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:58.688789 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7157ee-109f-4dca-a539-1afb4c0dbf7b None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:58.689066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7157ee-109f-4dca-a539-1afb4c0dbf7b None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:58.689731 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7157ee-109f-4dca-a539-1afb4c0dbf7b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:58.691205 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7157ee-109f-4dca-a539-1afb4c0dbf7b None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:58.693938 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: WARNING keystone.server.flask.common [None req-ca7157ee-109f-4dca-a539-1afb4c0dbf7b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 26 00:35:58.703419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-30442b61-0882-4f8a-87c8-0780b1c43770 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:58.704701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-30442b61-0882-4f8a-87c8-0780b1c43770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:58.734804 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.sql.core [None req-ca7157ee-109f-4dca-a539-1afb4c0dbf7b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 26 00:35:58.734804 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 26 00:35:58.734804 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [parameters: {'user_id': 'c2fb7b5a147b43b5acfe49a3afd16c63', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 26 00:35:58.734804 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88253) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 26 00:35:58.752699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-ca7157ee-109f-4dca-a539-1afb4c0dbf7b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 26 00:35:58.752699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:58.752699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 00:35:58.752699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 00:35:58.752699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 00:35:58.752699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 00:35:58.752699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 00:35:58.752699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 00:35:58.752699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 00:35:58.752699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application conn.query(q) Aug 26 00:35:58.752699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 00:35:58.752699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 00:35:58.752699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 00:35:58.752699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application result.read() Aug 26 00:35:58.752699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 00:35:58.753687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 00:35:58.753687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 00:35:58.753687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 00:35:58.753687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 00:35:58.753687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 00:35:58.753687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 00:35:58.753687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 00:35:58.753687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 26 00:35:58.753687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:58.753687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 26 00:35:58.753687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:58.753687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:58.753687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 26 00:35:58.753687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 26 00:35:58.753687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 26 00:35:58.753687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 26 00:35:58.753687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 00:35:58.753687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application next(self.gen) Aug 26 00:35:58.754709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 26 00:35:58.754709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application yield resource Aug 26 00:35:58.754709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 00:35:58.754709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application next(self.gen) Aug 26 00:35:58.754709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 26 00:35:58.754709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self.session.rollback() Aug 26 00:35:58.754709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 26 00:35:58.754709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self.force_reraise() Aug 26 00:35:58.754709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 26 00:35:58.754709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise self.value Aug 26 00:35:58.754709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 26 00:35:58.754709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 26 00:35:58.754709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 26 00:35:58.754709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application session.commit() Aug 26 00:35:58.754709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 26 00:35:58.754709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 26 00:35:58.754709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 26 00:35:58.755831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self._prepare_impl() Aug 26 00:35:58.755831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 26 00:35:58.755831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self.session.flush() Aug 26 00:35:58.755831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 26 00:35:58.755831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self._flush(objects) Aug 26 00:35:58.755831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 26 00:35:58.755831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 26 00:35:58.755831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 26 00:35:58.755831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application compat.raise_( Aug 26 00:35:58.755831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 00:35:58.755831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception Aug 26 00:35:58.755831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 26 00:35:58.755831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application flush_context.execute() Aug 26 00:35:58.755831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 26 00:35:58.755831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rec.execute(self) Aug 26 00:35:58.755831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 26 00:35:58.755831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 26 00:35:58.756827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 26 00:35:58.756827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 26 00:35:58.756827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 26 00:35:58.756827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 26 00:35:58.756827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 26 00:35:58.756827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 26 00:35:58.756827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 26 00:35:58.756827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 26 00:35:58.756827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 26 00:35:58.756827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 26 00:35:58.756827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 26 00:35:58.756827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 26 00:35:58.756827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 26 00:35:58.756827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 26 00:35:58.756827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 00:35:58.756827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception Aug 26 00:35:58.760223 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 00:35:58.760223 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 00:35:58.760223 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 00:35:58.760223 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 00:35:58.760223 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 00:35:58.760223 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 00:35:58.760223 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 00:35:58.760223 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application conn.query(q) Aug 26 00:35:58.760223 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 00:35:58.760223 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 00:35:58.760223 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 00:35:58.760223 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application result.read() Aug 26 00:35:58.760223 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 00:35:58.760223 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 00:35:58.760223 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 00:35:58.760223 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 00:35:58.762509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 00:35:58.762509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 00:35:58.762509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 00:35:58.762509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 00:35:58.762509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 26 00:35:58.762509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 26 00:35:58.762509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c2fb7b5a147b43b5acfe49a3afd16c63', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 26 00:35:58.762509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 26 00:35:58.762509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:58.762509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:35:58.762509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:58.762509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:58.762509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:58.762509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:58.762509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:58.764897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:58.764897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:58.764897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:58.764897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:58.764897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:58.764897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:58.764897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:58.764897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 26 00:35:58.764897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 26 00:35:58.764897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:58.764897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:58.764897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:58.764897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:58.764897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:58.764897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:58.764897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 26 00:35:58.764897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 26 00:35:58.768650 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 26 00:35:58.768650 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 26 00:35:58.768650 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 26 00:35:58.768650 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 26 00:35:58.768650 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 26 00:35:58.768650 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:58.768650 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 474/966] 10.222.0.59 () {60 vars in 1131 bytes} [Fri Aug 26 00:35:58 2022] POST /identity/v3/users => generated 160 bytes in 223 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:58.768650 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7157ee-109f-4dca-a539-1afb4c0dbf7b None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:58.797394 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 493/967] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:35:58.823657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:58.824654 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30442b61-0882-4f8a-87c8-0780b1c43770 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:58.880085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:58.884246 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7157ee-109f-4dca-a539-1afb4c0dbf7b None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:58.922135 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aef1de-410e-4742-8acf-e7112445b46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:58.922785 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aef1de-410e-4742-8acf-e7112445b46f None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:58.923038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aef1de-410e-4742-8acf-e7112445b46f None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:58.923320 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aef1de-410e-4742-8acf-e7112445b46f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:58.930751 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aef1de-410e-4742-8acf-e7112445b46f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:58.932407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aef1de-410e-4742-8acf-e7112445b46f None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:58.942850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30442b61-0882-4f8a-87c8-0780b1c43770 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:58.946081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-63aef1de-410e-4742-8acf-e7112445b46f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 00:35:58.946081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:58.946081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:58.946081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:58.946081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:58.946081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:58.946081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:58.946081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:58.946081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:58.946081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:58.946081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:58.946081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:58.946081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:35:58.946081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:35:58.946081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:35:58.946081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:35:58.947290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:58.947290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:58.947290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:35:58.947290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:58.947290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:35:58.947290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:35:58.947290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:58.947290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:58.947290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:58.947290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:58.947290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:58.947290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:58.947290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:58.947290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:58.947290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:58.947290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:58.947290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:58.947290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:58.948526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:58.948526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:58.948526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:35:58.948526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:35:58.948526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:35:58.948526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:35:58.948526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:35:58.948526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:35:58.948526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 00:35:58.948526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:58.948526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 475/968] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:35:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:58.952664 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aef1de-410e-4742-8acf-e7112445b46f None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:58.987068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968d009-c245-44b5-bdf1-dd0463b1f2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:58.988245 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968d009-c245-44b5-bdf1-dd0463b1f2b1 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:58.988779 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968d009-c245-44b5-bdf1-dd0463b1f2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:58.990087 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968d009-c245-44b5-bdf1-dd0463b1f2b1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:58.990913 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968d009-c245-44b5-bdf1-dd0463b1f2b1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:58.992641 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968d009-c245-44b5-bdf1-dd0463b1f2b1 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:59.005407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-f968d009-c245-44b5-bdf1-dd0463b1f2b1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 00:35:59.005407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:59.005407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 00:35:59.005407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 00:35:59.005407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 00:35:59.005407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 00:35:59.005407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 00:35:59.005407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:59.005407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 00:35:59.005407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:59.005407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:35:59.005407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:59.005407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:59.005407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:59.005407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:59.005407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:59.005407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:59.006722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:59.006722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:59.006722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:59.006722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:59.006722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:59.006722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:59.006722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 00:35:59.006722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 00:35:59.006722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 00:35:59.006722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 00:35:59.006722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:59.006722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:59.006722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:59.006722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:59.006722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:59.006722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:59.007811 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:59.007811 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:59.007811 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:59.007811 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:59.007811 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:59.007811 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:59.007811 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:59.007811 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:59.007811 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:59.007811 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:59.007811 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 00:35:59.007811 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:35:59.007811 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 00:35:59.007811 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:59.010653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 494/969] 10.222.0.59 () {56 vars in 1126 bytes} [Fri Aug 26 00:35:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:35:59.016330 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968d009-c245-44b5-bdf1-dd0463b1f2b1 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:59.074196 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:59.077318 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aef1de-410e-4742-8acf-e7112445b46f None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:59.112241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357dc4b-c5d3-4b69-bef0-c867328452d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:59.112878 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357dc4b-c5d3-4b69-bef0-c867328452d1 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:59.113156 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357dc4b-c5d3-4b69-bef0-c867328452d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:59.113427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357dc4b-c5d3-4b69-bef0-c867328452d1 None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:59.114127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357dc4b-c5d3-4b69-bef0-c867328452d1 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:59.114427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357dc4b-c5d3-4b69-bef0-c867328452d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:59.115864 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357dc4b-c5d3-4b69-bef0-c867328452d1 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:59.138104 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968d009-c245-44b5-bdf1-dd0463b1f2b1 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:59.149200 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:59.160518 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 476/970] 10.222.0.59 () {56 vars in 1101 bytes} [Fri Aug 26 00:35:58 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:59.160518 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357dc4b-c5d3-4b69-bef0-c867328452d1 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:59.180752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba309a-645b-42ac-99d9-4299625ecb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:59.181540 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba309a-645b-42ac-99d9-4299625ecb93 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:59.181954 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba309a-645b-42ac-99d9-4299625ecb93 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:59.182227 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba309a-645b-42ac-99d9-4299625ecb93 None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:59.182945 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba309a-645b-42ac-99d9-4299625ecb93 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:59.183339 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba309a-645b-42ac-99d9-4299625ecb93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:59.184932 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba309a-645b-42ac-99d9-4299625ecb93 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:59.191990 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 495/971] 10.222.0.59 () {56 vars in 1122 bytes} [Fri Aug 26 00:35:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:59.198168 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba309a-645b-42ac-99d9-4299625ecb93 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:59.293690 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357dc4b-c5d3-4b69-bef0-c867328452d1 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:59.312321 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba309a-645b-42ac-99d9-4299625ecb93 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:59.332373 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f8d3c-71b4-42a3-92f0-44de8b595679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:59.333062 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f8d3c-71b4-42a3-92f0-44de8b595679 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:59.333341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f8d3c-71b4-42a3-92f0-44de8b595679 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:59.333612 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f8d3c-71b4-42a3-92f0-44de8b595679 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:59.346089 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f8d3c-71b4-42a3-92f0-44de8b595679 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:35:59.348403 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c13ee-ace4-4d47-be6f-4d61e8d9de34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:59.349052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c13ee-ace4-4d47-be6f-4d61e8d9de34 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:59.349338 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c13ee-ace4-4d47-be6f-4d61e8d9de34 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:59.349596 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c13ee-ace4-4d47-be6f-4d61e8d9de34 None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:59.350225 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c13ee-ace4-4d47-be6f-4d61e8d9de34 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:59.350523 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c13ee-ace4-4d47-be6f-4d61e8d9de34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:59.351672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-037f8d3c-71b4-42a3-92f0-44de8b595679 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 00:35:59.351672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:59.351672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:59.351672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:59.351672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:59.351672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:59.351672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:59.351672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:59.351672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:59.351672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:59.351672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:59.351672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:59.351672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:35:59.351672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:35:59.351672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:35:59.351672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:35:59.352633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:35:59.352633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:35:59.352633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:35:59.352633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:35:59.352633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:59.352633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:59.352633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:59.352633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:59.352633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:59.352633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:59.352633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:59.352633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:59.352633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:59.352633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:59.352633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:59.352633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:59.353541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:59.353541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:59.353541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:59.353541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:59.353541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:35:59.353541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:35:59.353541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:35:59.353541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:35:59.353541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:35:59.353541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:35:59.353541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 00:35:59.353541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:35:59.354590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c13ee-ace4-4d47-be6f-4d61e8d9de34 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:59.356925 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 477/972] 10.222.0.59 () {56 vars in 1120 bytes} [Fri Aug 26 00:35:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:59.358533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06ff7e-57a3-43f5-ad67-9ff7e12c2f6f None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:59.358818 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06ff7e-57a3-43f5-ad67-9ff7e12c2f6f None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:59.359089 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06ff7e-57a3-43f5-ad67-9ff7e12c2f6f None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:59.359650 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 496/973] 10.222.0.59 () {56 vars in 1122 bytes} [Fri Aug 26 00:35:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:35:59.360498 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 478/974] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:35:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:35:59.362481 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c13ee-ace4-4d47-be6f-4d61e8d9de34 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:59.365639 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e57980-92de-40a6-8527-5923855f1777 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:59.365919 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e57980-92de-40a6-8527-5923855f1777 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:59.366188 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e57980-92de-40a6-8527-5923855f1777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:59.385614 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-43e57980-92de-40a6-8527-5923855f1777 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:35:59.387216 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-43e57980-92de-40a6-8527-5923855f1777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:35:59.407260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:59.415993 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:59.462359 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 479/975] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:35:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:35:59.463135 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e57980-92de-40a6-8527-5923855f1777 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:59.484179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c13ee-ace4-4d47-be6f-4d61e8d9de34 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:59.519895 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b2cbc6-e097-462c-818a-db326186edc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:59.520529 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2cbc6-e097-462c-818a-db326186edc2 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:59.520812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2cbc6-e097-462c-818a-db326186edc2 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:59.521081 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2cbc6-e097-462c-818a-db326186edc2 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:59.521861 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b2cbc6-e097-462c-818a-db326186edc2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:59.522278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b2cbc6-e097-462c-818a-db326186edc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:59.523641 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b2cbc6-e097-462c-818a-db326186edc2 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:59.537002 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 497/976] 10.222.0.59 () {56 vars in 1115 bytes} [Fri Aug 26 00:35:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:35:59.537871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b2cbc6-e097-462c-818a-db326186edc2 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:59.564334 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e57980-92de-40a6-8527-5923855f1777 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:59.599675 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7234df50-2035-4bae-8939-6a3b3b0d9001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:59.600281 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234df50-2035-4bae-8939-6a3b3b0d9001 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:59.600592 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234df50-2035-4bae-8939-6a3b3b0d9001 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:59.600865 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234df50-2035-4bae-8939-6a3b3b0d9001 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:59.601567 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7234df50-2035-4bae-8939-6a3b3b0d9001 None admin] RBAC: Adding query filter params (interface=admin, service_id=110fe8414d8645a09d1f5d94751b0965, region_id=RegionOne) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:59.601875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7234df50-2035-4bae-8939-6a3b3b0d9001 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:59.603259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7234df50-2035-4bae-8939-6a3b3b0d9001 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:59.609424 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 480/977] 10.222.0.59 () {56 vars in 1249 bytes} [Fri Aug 26 00:35:59 2022] GET /identity/v3/endpoints?service_id=110fe8414d8645a09d1f5d94751b0965&interface=admin®ion_id=RegionOne => generated 193 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:35:59.615213 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7234df50-2035-4bae-8939-6a3b3b0d9001 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:59.632205 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:59.639132 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:59.642467 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b2cbc6-e097-462c-818a-db326186edc2 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:59.680606 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f5d07-3d1a-4031-b6c3-2584e686be12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:59.681236 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f5d07-3d1a-4031-b6c3-2584e686be12 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:59.681511 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f5d07-3d1a-4031-b6c3-2584e686be12 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:59.681781 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f5d07-3d1a-4031-b6c3-2584e686be12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:59.691620 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f5d07-3d1a-4031-b6c3-2584e686be12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:59.693103 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f5d07-3d1a-4031-b6c3-2584e686be12 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:59.708289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-4f2f5d07-3d1a-4031-b6c3-2584e686be12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:59.708289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:35:59.708289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:35:59.708289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:35:59.708289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:35:59.708289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:35:59.708289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:35:59.708289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:35:59.708289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:35:59.708289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:35:59.708289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:35:59.708289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:35:59.708289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:35:59.708289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:35:59.708289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:35:59.708289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:35:59.709497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:35:59.709497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:35:59.709497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:35:59.709497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:35:59.709497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:35:59.709497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:35:59.709497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:35:59.709497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:35:59.709497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:35:59.709497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:35:59.709497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:35:59.709497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:35:59.709497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:35:59.709497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:35:59.709497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:35:59.709497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:35:59.709497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:35:59.710414 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:35:59.710414 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:35:59.710414 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:35:59.710414 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 498/978] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:35:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:35:59.715239 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f5d07-3d1a-4031-b6c3-2584e686be12 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:59.730677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7234df50-2035-4bae-8939-6a3b3b0d9001 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:59.764252 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb28515-186e-4578-a735-f5ca8f74ce75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:59.764879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb28515-186e-4578-a735-f5ca8f74ce75 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:59.765153 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb28515-186e-4578-a735-f5ca8f74ce75 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:59.765421 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb28515-186e-4578-a735-f5ca8f74ce75 None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:59.766018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb28515-186e-4578-a735-f5ca8f74ce75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:59.767364 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb28515-186e-4578-a735-f5ca8f74ce75 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:59.773280 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 481/979] 10.222.0.59 () {56 vars in 1087 bytes} [Fri Aug 26 00:35:59 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 00:35:59.773922 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb28515-186e-4578-a735-f5ca8f74ce75 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:59.824517 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f5d07-3d1a-4031-b6c3-2584e686be12 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:59.864212 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cddb3e-6f98-49b5-8ad3-e7ccacb96b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:59.865165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cddb3e-6f98-49b5-8ad3-e7ccacb96b1d None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:59.865165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cddb3e-6f98-49b5-8ad3-e7ccacb96b1d None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:59.865406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cddb3e-6f98-49b5-8ad3-e7ccacb96b1d None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:59.865858 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:59.866049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cddb3e-6f98-49b5-8ad3-e7ccacb96b1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:35:59.866360 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cddb3e-6f98-49b5-8ad3-e7ccacb96b1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:59.867951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cddb3e-6f98-49b5-8ad3-e7ccacb96b1d None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:59.881689 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 499/980] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:35:59.887643 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cddb3e-6f98-49b5-8ad3-e7ccacb96b1d None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:35:59.897743 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb28515-186e-4578-a735-f5ca8f74ce75 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:35:59.903847 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:35:59.939918 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c597e85-67d9-466a-acdb-7ccc1d770df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:35:59.940605 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c597e85-67d9-466a-acdb-7ccc1d770df3 None admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:35:59.940894 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c597e85-67d9-466a-acdb-7ccc1d770df3 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:35:59.941174 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c597e85-67d9-466a-acdb-7ccc1d770df3 None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:35:59.941831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c597e85-67d9-466a-acdb-7ccc1d770df3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:35:59.943207 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c597e85-67d9-466a-acdb-7ccc1d770df3 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:35:59.994269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cddb3e-6f98-49b5-8ad3-e7ccacb96b1d None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:00.020580 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 482/981] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Aug 26 00:35:59 2022] POST /identity/v3/services => generated 242 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:36:00.039949 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3de8cc-2916-4412-ae42-24bce2596727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:00.040743 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3de8cc-2916-4412-ae42-24bce2596727 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:00.041140 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3de8cc-2916-4412-ae42-24bce2596727 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:00.041419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3de8cc-2916-4412-ae42-24bce2596727 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:00.050957 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3de8cc-2916-4412-ae42-24bce2596727 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:00.052882 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3de8cc-2916-4412-ae42-24bce2596727 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:00.074261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-3b3de8cc-2916-4412-ae42-24bce2596727 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 00:36:00.074261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:00.074261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:00.074261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:00.074261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:00.074261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:00.074261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:00.074261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:00.074261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:00.074261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:00.074261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:00.074261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:00.074261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:36:00.074261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:36:00.074261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:36:00.074261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:36:00.075351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:00.075351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:00.075351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:36:00.075351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:36:00.075351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:36:00.075351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:36:00.075351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:00.075351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:00.075351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:00.075351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:00.075351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:00.075351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:00.075351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:00.075351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:00.075351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:00.075351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:00.075351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:00.075351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:00.076453 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:00.076453 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:00.076453 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:36:00.076453 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:36:00.076453 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:36:00.076453 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:36:00.076453 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:36:00.076453 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:36:00.076453 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 00:36:00.076453 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:36:00.080453 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 500/982] 10.222.0.59 () {56 vars in 1117 bytes} [Fri Aug 26 00:35:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:36:00.081769 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c597e85-67d9-466a-acdb-7ccc1d770df3 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:00.193160 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c597e85-67d9-466a-acdb-7ccc1d770df3 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:00.228431 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00bc7c-9488-4a22-a211-c86dea7448d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:00.229079 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00bc7c-9488-4a22-a211-c86dea7448d6 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:00.229364 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00bc7c-9488-4a22-a211-c86dea7448d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:00.229641 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00bc7c-9488-4a22-a211-c86dea7448d6 None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:00.230382 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb00bc7c-9488-4a22-a211-c86dea7448d6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:00.230690 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb00bc7c-9488-4a22-a211-c86dea7448d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:00.232113 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb00bc7c-9488-4a22-a211-c86dea7448d6 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:00.235307 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:00.251334 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 483/983] 10.222.0.59 () {56 vars in 1149 bytes} [Fri Aug 26 00:36:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:00.256728 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3de8cc-2916-4412-ae42-24bce2596727 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:00.327667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:00.361693 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3de8cc-2916-4412-ae42-24bce2596727 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:00.397024 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b8ce9-cba1-4c6e-b7f5-e2268316fba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:00.397779 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b8ce9-cba1-4c6e-b7f5-e2268316fba8 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:00.398172 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b8ce9-cba1-4c6e-b7f5-e2268316fba8 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:00.398538 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b8ce9-cba1-4c6e-b7f5-e2268316fba8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:00.477842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3b8ce9-cba1-4c6e-b7f5-e2268316fba8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:00.479295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3b8ce9-cba1-4c6e-b7f5-e2268316fba8 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:00.546175 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:00.584244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 501/984] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:36:00 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 00:36:00.585549 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00bc7c-9488-4a22-a211-c86dea7448d6 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:00.686941 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00bc7c-9488-4a22-a211-c86dea7448d6 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:00.720862 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25a6bd-2cc2-494b-87fd-f504c0da8860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:00.721590 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25a6bd-2cc2-494b-87fd-f504c0da8860 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:00.721943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25a6bd-2cc2-494b-87fd-f504c0da8860 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:00.722287 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25a6bd-2cc2-494b-87fd-f504c0da8860 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:00.732558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f25a6bd-2cc2-494b-87fd-f504c0da8860 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:00.734629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f25a6bd-2cc2-494b-87fd-f504c0da8860 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:00.749408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-9f25a6bd-2cc2-494b-87fd-f504c0da8860 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:36:00.749408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:00.749408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:00.749408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:00.749408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:00.749408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:00.749408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:00.749408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:00.749408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:00.749408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:00.749408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:00.749408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:00.749408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:36:00.749408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:36:00.749408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:36:00.749408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:36:00.750410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:00.750410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:00.750410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:00.750410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:00.750410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:00.750410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:00.750410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:00.750410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:00.750410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:00.750410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:00.750410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:00.750410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:00.750410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:00.750410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:00.750410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:00.750410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:00.750410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:36:00.751406 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:36:00.751406 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:36:00.751406 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:36:00.751406 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 484/985] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:36:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:36:00.756040 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b8ce9-cba1-4c6e-b7f5-e2268316fba8 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:00.854236 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b8ce9-cba1-4c6e-b7f5-e2268316fba8 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:00.890189 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a884fe5-654e-4b53-9e13-55ad41cf378d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:00.890795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a884fe5-654e-4b53-9e13-55ad41cf378d None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:00.891072 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a884fe5-654e-4b53-9e13-55ad41cf378d None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:00.891342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a884fe5-654e-4b53-9e13-55ad41cf378d None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:00.891976 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a884fe5-654e-4b53-9e13-55ad41cf378d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:00.892275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a884fe5-654e-4b53-9e13-55ad41cf378d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:00.893661 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a884fe5-654e-4b53-9e13-55ad41cf378d None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:00.909638 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 502/986] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:36:00.910833 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25a6bd-2cc2-494b-87fd-f504c0da8860 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:00.959316 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:01.014436 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25a6bd-2cc2-494b-87fd-f504c0da8860 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:01.048940 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409886aa-6305-45c3-a032-7265016d0412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:01.049562 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409886aa-6305-45c3-a032-7265016d0412 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:01.049848 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409886aa-6305-45c3-a032-7265016d0412 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:01.050259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409886aa-6305-45c3-a032-7265016d0412 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:01.056943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409886aa-6305-45c3-a032-7265016d0412 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:01.058416 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409886aa-6305-45c3-a032-7265016d0412 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:01.070048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-409886aa-6305-45c3-a032-7265016d0412 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:36:01.070048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:01.070048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:01.070048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:01.070048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:01.070048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:01.070048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:01.070048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:01.070048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:01.070048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:01.070048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:01.070048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:01.070048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:36:01.070048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:36:01.070048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:36:01.071145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:36:01.071145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:01.071145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:01.071145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:01.071145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:01.071145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:01.071145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:01.071145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:01.071145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:01.071145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:01.071145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:01.071145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:01.071145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:01.071145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:01.071145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:01.071145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:01.071145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:01.072684 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:36:01.072684 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:36:01.072684 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:36:01.072684 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:36:01.072684 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:36:01.072684 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:36:01.072684 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:36:01.072684 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:36:01.072684 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 485/987] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:36:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:36:01.076914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a884fe5-654e-4b53-9e13-55ad41cf378d None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:01.106306 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:01.180438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a884fe5-654e-4b53-9e13-55ad41cf378d None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:01.213875 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363041c-1b18-4e2c-a1f3-b78077db0155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:01.214483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363041c-1b18-4e2c-a1f3-b78077db0155 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:01.214761 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363041c-1b18-4e2c-a1f3-b78077db0155 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:01.215028 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363041c-1b18-4e2c-a1f3-b78077db0155 None admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:01.215707 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363041c-1b18-4e2c-a1f3-b78077db0155 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:01.216008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363041c-1b18-4e2c-a1f3-b78077db0155 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:01.217993 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363041c-1b18-4e2c-a1f3-b78077db0155 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:01.234855 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 503/988] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:36:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:01.236121 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409886aa-6305-45c3-a032-7265016d0412 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:01.334178 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409886aa-6305-45c3-a032-7265016d0412 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:01.341124 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:01.368329 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6bc0b1-7ad9-4dc0-96b3-b97981fef651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:01.368934 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bc0b1-7ad9-4dc0-96b3-b97981fef651 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:01.369211 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bc0b1-7ad9-4dc0-96b3-b97981fef651 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:01.369530 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bc0b1-7ad9-4dc0-96b3-b97981fef651 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:01.370255 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6bc0b1-7ad9-4dc0-96b3-b97981fef651 None admin] RBAC: Adding query filter params (role.id=91940e122e2f46e3818c5f11f4fca9e9, scope.project.id=fde7ac3ed472433ab3bba56e5e091510, user.id=bbd9bbb083ab4f5e84e135ec5103cdae) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:01.370558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6bc0b1-7ad9-4dc0-96b3-b97981fef651 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:01.371977 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6bc0b1-7ad9-4dc0-96b3-b97981fef651 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:01.384566 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 486/989] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:36:01 2022] GET /identity/v3/role_assignments?user.id=bbd9bbb083ab4f5e84e135ec5103cdae&scope.project.id=fde7ac3ed472433ab3bba56e5e091510&role.id=91940e122e2f46e3818c5f11f4fca9e9 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:01.712568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e2385-c65b-4231-875b-f13652c1e828 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:01.713076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e2385-c65b-4231-875b-f13652c1e828 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:01.713076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e2385-c65b-4231-875b-f13652c1e828 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:01.714524 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 504/990] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:36:01.719780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe877a-10eb-44de-979e-afe5a888c4ea None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:01.720073 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe877a-10eb-44de-979e-afe5a888c4ea None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:01.720350 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe877a-10eb-44de-979e-afe5a888c4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:01.740047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-f2fe877a-10eb-44de-979e-afe5a888c4ea None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:01.741276 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-f2fe877a-10eb-44de-979e-afe5a888c4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:01.814997 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 487/991] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:36:01.965410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860409a-2f13-4c5f-a70d-08df1361e767 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:01.965710 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860409a-2f13-4c5f-a70d-08df1361e767 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:01.965887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860409a-2f13-4c5f-a70d-08df1361e767 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:01.967388 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 505/992] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:36:01.972484 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c3a0f-b1ba-41e8-952d-62fc8cf944c8 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:01.972790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c3a0f-b1ba-41e8-952d-62fc8cf944c8 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:01.973073 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c3a0f-b1ba-41e8-952d-62fc8cf944c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:01.990733 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-276c3a0f-b1ba-41e8-952d-62fc8cf944c8 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:01.991913 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-276c3a0f-b1ba-41e8-952d-62fc8cf944c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:02.069896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 488/993] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:36:02.084785 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3860409a-2f13-4c5f-a70d-08df1361e767 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:02.174204 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:02.187949 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3860409a-2f13-4c5f-a70d-08df1361e767 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:02.223585 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da536dc6-acc0-4b49-85e4-d50b88d04dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:02.224144 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da536dc6-acc0-4b49-85e4-d50b88d04dc0 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:02.224425 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da536dc6-acc0-4b49-85e4-d50b88d04dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:02.224729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da536dc6-acc0-4b49-85e4-d50b88d04dc0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:02.225400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da536dc6-acc0-4b49-85e4-d50b88d04dc0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:02.226921 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da536dc6-acc0-4b49-85e4-d50b88d04dc0 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:02.236025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-da536dc6-acc0-4b49-85e4-d50b88d04dc0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 00:36:02.236025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:02.236025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 00:36:02.236025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 00:36:02.236025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 00:36:02.236025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 00:36:02.236025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 00:36:02.236025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:36:02.236025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 00:36:02.236025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:36:02.236025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:36:02.236025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:36:02.236025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:02.236025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:02.236025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:02.236025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:02.236025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:02.237246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:02.237246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:02.237246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:02.237246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:02.237246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:02.237246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:02.237246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 00:36:02.237246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 00:36:02.237246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 00:36:02.237246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 00:36:02.237246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:02.237246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:02.237246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:02.237246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:02.237246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:02.237246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:02.238165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:02.238165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:02.238165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:02.238165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:02.238165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:02.238165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:02.238165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:02.238165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:02.238165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:02.238165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:02.238165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 00:36:02.238165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:36:02.238165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 00:36:02.238165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:36:02.238165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 506/994] 10.222.0.59 () {56 vars in 1126 bytes} [Fri Aug 26 00:36:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:36:02.245248 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c3a0f-b1ba-41e8-952d-62fc8cf944c8 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:02.276599 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:02.345581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da536dc6-acc0-4b49-85e4-d50b88d04dc0 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:02.347847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c3a0f-b1ba-41e8-952d-62fc8cf944c8 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:02.389729 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e229e74-0663-4bd5-bb04-2313d90c2a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:02.390401 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e229e74-0663-4bd5-bb04-2313d90c2a03 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:02.390675 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e229e74-0663-4bd5-bb04-2313d90c2a03 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:02.390951 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e229e74-0663-4bd5-bb04-2313d90c2a03 None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:02.391598 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e229e74-0663-4bd5-bb04-2313d90c2a03 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:02.391900 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e229e74-0663-4bd5-bb04-2313d90c2a03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:02.393474 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e229e74-0663-4bd5-bb04-2313d90c2a03 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:02.399957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 489/995] 10.222.0.59 () {56 vars in 1122 bytes} [Fri Aug 26 00:36:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:02.407826 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e229e74-0663-4bd5-bb04-2313d90c2a03 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:02.474842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da536dc6-acc0-4b49-85e4-d50b88d04dc0 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:02.511239 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:02.514805 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a99f1-8cab-42ac-a00e-97014ecc6228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:02.515599 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a99f1-8cab-42ac-a00e-97014ecc6228 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:02.516050 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a99f1-8cab-42ac-a00e-97014ecc6228 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:02.516486 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a99f1-8cab-42ac-a00e-97014ecc6228 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:02.517153 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a99f1-8cab-42ac-a00e-97014ecc6228 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:02.518519 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a99f1-8cab-42ac-a00e-97014ecc6228 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:02.518688 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e229e74-0663-4bd5-bb04-2313d90c2a03 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:02.527979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-943a99f1-8cab-42ac-a00e-97014ecc6228 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 00:36:02.527979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:02.527979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 00:36:02.527979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 00:36:02.527979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 00:36:02.527979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 00:36:02.527979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 00:36:02.527979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:36:02.527979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 00:36:02.527979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:36:02.527979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:36:02.527979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:36:02.527979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:02.527979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:02.527979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:02.527979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:02.527979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:02.529763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:02.529763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:02.529763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:02.529763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:02.529763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:02.529763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:02.529763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 00:36:02.529763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 00:36:02.529763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 00:36:02.529763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 00:36:02.529763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:02.529763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:02.529763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:02.529763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:02.529763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:02.529763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:02.530719 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:02.530719 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:02.530719 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:02.530719 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:02.530719 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:02.530719 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:02.530719 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:02.530719 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:02.530719 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:02.530719 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:02.530719 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 00:36:02.530719 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:36:02.530719 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 00:36:02.530719 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:36:02.530719 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 507/996] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:36:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:36:02.536116 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a99f1-8cab-42ac-a00e-97014ecc6228 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:02.556845 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:02.559534 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb430f09-c949-4ff6-9a2a-12d48dde164d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:02.560211 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb430f09-c949-4ff6-9a2a-12d48dde164d None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:02.560511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb430f09-c949-4ff6-9a2a-12d48dde164d None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:02.560873 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb430f09-c949-4ff6-9a2a-12d48dde164d None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:02.561509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb430f09-c949-4ff6-9a2a-12d48dde164d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:02.561814 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb430f09-c949-4ff6-9a2a-12d48dde164d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:02.563242 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb430f09-c949-4ff6-9a2a-12d48dde164d None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:02.568362 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 490/997] 10.222.0.59 () {56 vars in 1122 bytes} [Fri Aug 26 00:36:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:02.574557 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb430f09-c949-4ff6-9a2a-12d48dde164d None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:02.646236 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a99f1-8cab-42ac-a00e-97014ecc6228 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:02.679450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb430f09-c949-4ff6-9a2a-12d48dde164d None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:02.682575 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc08346-292f-4985-9055-10a8769fa6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:02.683291 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc08346-292f-4985-9055-10a8769fa6d3 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:02.683572 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc08346-292f-4985-9055-10a8769fa6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:02.683843 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc08346-292f-4985-9055-10a8769fa6d3 None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:02.684562 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc08346-292f-4985-9055-10a8769fa6d3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:02.684868 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc08346-292f-4985-9055-10a8769fa6d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:02.686790 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc08346-292f-4985-9055-10a8769fa6d3 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:02.691917 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 508/998] 10.222.0.59 () {56 vars in 1112 bytes} [Fri Aug 26 00:36:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:02.697714 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc08346-292f-4985-9055-10a8769fa6d3 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:02.722722 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280c292-4b4a-4b1b-ab2c-aef850e98704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:02.723486 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280c292-4b4a-4b1b-ab2c-aef850e98704 None admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:02.723761 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280c292-4b4a-4b1b-ab2c-aef850e98704 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:02.724085 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280c292-4b4a-4b1b-ab2c-aef850e98704 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:02.724779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280c292-4b4a-4b1b-ab2c-aef850e98704 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:02.726560 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280c292-4b4a-4b1b-ab2c-aef850e98704 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:02.777496 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:02.804518 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 491/999] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:36:02 2022] POST /identity/v3/endpoints => generated 325 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:36:02.804735 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc08346-292f-4985-9055-10a8769fa6d3 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:02.846510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d96a1-61c5-4fc3-93fd-106d4b448971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:02.847195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d96a1-61c5-4fc3-93fd-106d4b448971 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:02.847479 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d96a1-61c5-4fc3-93fd-106d4b448971 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:02.847752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d96a1-61c5-4fc3-93fd-106d4b448971 None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:02.848401 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d96a1-61c5-4fc3-93fd-106d4b448971 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:02.848733 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d96a1-61c5-4fc3-93fd-106d4b448971 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:02.850241 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d96a1-61c5-4fc3-93fd-106d4b448971 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:02.861795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 509/1000] 10.222.0.59 () {56 vars in 1112 bytes} [Fri Aug 26 00:36:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:36:02.863572 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280c292-4b4a-4b1b-ab2c-aef850e98704 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:02.885018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:02.975064 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280c292-4b4a-4b1b-ab2c-aef850e98704 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:03.011806 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0938149-3e01-45c1-998d-fd380dd02c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:03.012344 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0938149-3e01-45c1-998d-fd380dd02c42 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:03.012657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0938149-3e01-45c1-998d-fd380dd02c42 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:03.012932 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0938149-3e01-45c1-998d-fd380dd02c42 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:03.013636 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0938149-3e01-45c1-998d-fd380dd02c42 None admin] RBAC: Adding query filter params (interface=public, service_id=f87e4bac1ee94cd5a8ce718292106b5c, region_id=RegionOne) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:03.014478 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0938149-3e01-45c1-998d-fd380dd02c42 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:03.015850 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0938149-3e01-45c1-998d-fd380dd02c42 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:03.021429 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 492/1001] 10.222.0.59 () {56 vars in 1251 bytes} [Fri Aug 26 00:36:02 2022] GET /identity/v3/endpoints?service_id=f87e4bac1ee94cd5a8ce718292106b5c&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:03.027411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d96a1-61c5-4fc3-93fd-106d4b448971 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:03.131523 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d96a1-61c5-4fc3-93fd-106d4b448971 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:03.165157 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49762b5-e6ea-44df-bf6f-369a08ffd1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:03.165773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49762b5-e6ea-44df-bf6f-369a08ffd1ed None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:03.166051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49762b5-e6ea-44df-bf6f-369a08ffd1ed None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:03.166328 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49762b5-e6ea-44df-bf6f-369a08ffd1ed None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:03.166934 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49762b5-e6ea-44df-bf6f-369a08ffd1ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:03.168303 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49762b5-e6ea-44df-bf6f-369a08ffd1ed None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:03.180519 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 510/1002] 10.222.0.59 () {56 vars in 1087 bytes} [Fri Aug 26 00:36:03 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:36:03.321354 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5d7f9-7096-44b5-b07b-3c8f9fdce741 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:03.321870 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5d7f9-7096-44b5-b07b-3c8f9fdce741 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:03.321870 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5d7f9-7096-44b5-b07b-3c8f9fdce741 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:03.323399 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 493/1003] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:36:03.328610 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7faeaf-7796-4741-8165-c4a150cb78c7 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:03.328990 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7faeaf-7796-4741-8165-c4a150cb78c7 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:03.329377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7faeaf-7796-4741-8165-c4a150cb78c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:03.336766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:03.347977 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-0c7faeaf-7796-4741-8165-c4a150cb78c7 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:03.349212 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-0c7faeaf-7796-4741-8165-c4a150cb78c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:03.419143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 511/1004] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:03 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:36:03.686466 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5d7f9-7096-44b5-b07b-3c8f9fdce741 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:03.790470 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:03.809671 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5d7f9-7096-44b5-b07b-3c8f9fdce741 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:03.845967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b86b2-b479-495f-98a6-abbe22f8b44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:03.846949 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b86b2-b479-495f-98a6-abbe22f8b44d None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:03.847467 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b86b2-b479-495f-98a6-abbe22f8b44d None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:03.847943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b86b2-b479-495f-98a6-abbe22f8b44d None admin] PATH_INFO: `/v3/services/image` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:03.848785 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b86b2-b479-495f-98a6-abbe22f8b44d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:03.850346 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b86b2-b479-495f-98a6-abbe22f8b44d None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:03.859557 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-1c5b86b2-b479-495f-98a6-abbe22f8b44d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 00:36:03.859557 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:03.859557 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 00:36:03.859557 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 00:36:03.859557 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 00:36:03.859557 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 00:36:03.859557 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 00:36:03.859557 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:36:03.859557 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 00:36:03.859557 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:36:03.859557 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:36:03.859557 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:36:03.859557 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:03.859557 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:03.859557 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:03.859557 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:03.859557 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:03.860729 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:03.860729 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:03.860729 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:03.860729 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:03.860729 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:03.860729 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:03.860729 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 00:36:03.860729 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 00:36:03.860729 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 00:36:03.860729 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 00:36:03.860729 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:03.860729 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:03.860729 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:03.860729 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:03.860729 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:03.860729 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:03.861611 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:03.861611 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:03.861611 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:03.861611 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:03.861611 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:03.861611 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:03.861611 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:03.861611 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:03.861611 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:03.861611 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:03.861611 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 00:36:03.861611 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:36:03.861611 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 00:36:03.861611 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:36:03.863172 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 494/1005] 10.222.0.59 () {56 vars in 1105 bytes} [Fri Aug 26 00:36:03 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:36:03.867642 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7faeaf-7796-4741-8165-c4a150cb78c7 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:03.970021 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7faeaf-7796-4741-8165-c4a150cb78c7 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:04.002659 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d55daee-7c66-45a0-a4d1-6d5f90662bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:04.003265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55daee-7c66-45a0-a4d1-6d5f90662bf1 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:04.003541 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55daee-7c66-45a0-a4d1-6d5f90662bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:04.003808 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55daee-7c66-45a0-a4d1-6d5f90662bf1 None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:04.004551 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d55daee-7c66-45a0-a4d1-6d5f90662bf1 None admin] RBAC: Adding query filter params (name=image) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:04.004748 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d55daee-7c66-45a0-a4d1-6d5f90662bf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:04.006099 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d55daee-7c66-45a0-a4d1-6d5f90662bf1 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:04.007173 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:04.017341 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 512/1006] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:36:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:04.019431 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b86b2-b479-495f-98a6-abbe22f8b44d None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:04.127221 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b86b2-b479-495f-98a6-abbe22f8b44d None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:04.165769 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c610059-f102-47c2-829b-f20dc28c0225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:04.166382 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c610059-f102-47c2-829b-f20dc28c0225 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:04.166660 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c610059-f102-47c2-829b-f20dc28c0225 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:04.166933 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c610059-f102-47c2-829b-f20dc28c0225 None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:04.167560 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c610059-f102-47c2-829b-f20dc28c0225 None admin] RBAC: Adding query filter params (type=image) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:04.167865 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c610059-f102-47c2-829b-f20dc28c0225 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:04.169248 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c610059-f102-47c2-829b-f20dc28c0225 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:04.180548 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 495/1007] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:36:04 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:04.791873 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07f3eb-3c70-4cc3-94e9-2c14dd2fa6cf None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:04.792068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07f3eb-3c70-4cc3-94e9-2c14dd2fa6cf None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:04.792342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07f3eb-3c70-4cc3-94e9-2c14dd2fa6cf None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:04.793879 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 513/1008] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:36:04.798943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a65e4-b885-488c-a02c-3978ec798948 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:04.799397 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a65e4-b885-488c-a02c-3978ec798948 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:04.799477 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a65e4-b885-488c-a02c-3978ec798948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:04.817431 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-c78a65e4-b885-488c-a02c-3978ec798948 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:04.818614 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-c78a65e4-b885-488c-a02c-3978ec798948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:04.822099 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:04.886095 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 496/1009] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:04 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:36:05.099554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb50b3-e149-4960-8936-fda2b4be2c40 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:05.100096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb50b3-e149-4960-8936-fda2b4be2c40 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:05.100096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb50b3-e149-4960-8936-fda2b4be2c40 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:05.101630 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 514/1010] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:36:05.106598 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff8fba-0d09-4af8-ad3c-7064a2773ad9 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:05.106889 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff8fba-0d09-4af8-ad3c-7064a2773ad9 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:05.108720 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff8fba-0d09-4af8-ad3c-7064a2773ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:05.149755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-a4ff8fba-0d09-4af8-ad3c-7064a2773ad9 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:05.150941 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-a4ff8fba-0d09-4af8-ad3c-7064a2773ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:05.157976 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb50b3-e149-4960-8936-fda2b4be2c40 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:05.220923 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 497/1011] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:05 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:36:05.262747 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb50b3-e149-4960-8936-fda2b4be2c40 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:05.297063 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99be20-000f-48eb-b70b-70eed44934e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:05.297692 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99be20-000f-48eb-b70b-70eed44934e8 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:05.297971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99be20-000f-48eb-b70b-70eed44934e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:05.298243 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99be20-000f-48eb-b70b-70eed44934e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:05.299353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99be20-000f-48eb-b70b-70eed44934e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:05.300809 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99be20-000f-48eb-b70b-70eed44934e8 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:05.302566 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 515/1012] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:36:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:36:05.308071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff8fba-0d09-4af8-ad3c-7064a2773ad9 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:05.376400 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:05.409754 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff8fba-0d09-4af8-ad3c-7064a2773ad9 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:05.448725 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee00d8e-e177-48ee-9b19-34db75cfaab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:05.449334 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee00d8e-e177-48ee-9b19-34db75cfaab2 None admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:05.449614 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee00d8e-e177-48ee-9b19-34db75cfaab2 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:05.449888 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee00d8e-e177-48ee-9b19-34db75cfaab2 None admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:05.450544 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee00d8e-e177-48ee-9b19-34db75cfaab2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:05.451988 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee00d8e-e177-48ee-9b19-34db75cfaab2 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:05.485976 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99be20-000f-48eb-b70b-70eed44934e8 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:05.544807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 498/1013] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Aug 26 00:36:05 2022] POST /identity/v3/projects => generated 311 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:36:05.556640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:05.596061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99be20-000f-48eb-b70b-70eed44934e8 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:05.634571 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e3f77-d40e-499c-a885-58382a2d38fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:05.635137 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e3f77-d40e-499c-a885-58382a2d38fb None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:05.635418 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e3f77-d40e-499c-a885-58382a2d38fb None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:05.635690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e3f77-d40e-499c-a885-58382a2d38fb None admin] PATH_INFO: `/v3/services/compute` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:05.636283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e3f77-d40e-499c-a885-58382a2d38fb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:05.637714 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e3f77-d40e-499c-a885-58382a2d38fb None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:05.647164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-b07e3f77-d40e-499c-a885-58382a2d38fb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 00:36:05.647164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:05.647164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 00:36:05.647164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 00:36:05.647164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 00:36:05.647164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 00:36:05.647164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 00:36:05.647164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:36:05.647164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 00:36:05.647164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:36:05.647164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:36:05.647164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:36:05.647164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:05.647164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:05.647164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:05.647164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:05.647164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:05.648250 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:05.648250 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:05.648250 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:05.648250 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:05.648250 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:05.648250 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:05.648250 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 00:36:05.648250 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 00:36:05.648250 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 00:36:05.648250 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 00:36:05.648250 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:05.648250 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:05.648250 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:05.648250 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:05.648250 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:05.648250 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:05.654023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:05.654023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:05.654023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:05.654023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:05.654023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:05.654023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:05.654023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:05.654023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:05.654023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:05.654023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:05.654023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 00:36:05.654023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:36:05.654023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 00:36:05.654023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:36:05.654023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 516/1014] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:36:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:36:05.655810 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee00d8e-e177-48ee-9b19-34db75cfaab2 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:05.763510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee00d8e-e177-48ee-9b19-34db75cfaab2 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:05.787863 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:05.799849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9adfcfd-ec24-45be-9f58-4579bf0e9390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:05.807421 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9adfcfd-ec24-45be-9f58-4579bf0e9390 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:05.807421 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9adfcfd-ec24-45be-9f58-4579bf0e9390 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:05.807421 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9adfcfd-ec24-45be-9f58-4579bf0e9390 None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:05.807421 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9adfcfd-ec24-45be-9f58-4579bf0e9390 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:05.807421 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9adfcfd-ec24-45be-9f58-4579bf0e9390 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:05.807421 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9adfcfd-ec24-45be-9f58-4579bf0e9390 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:05.812763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 499/1015] 10.222.0.59 () {56 vars in 1112 bytes} [Fri Aug 26 00:36:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:05.814283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e3f77-d40e-499c-a885-58382a2d38fb None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:05.918393 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e3f77-d40e-499c-a885-58382a2d38fb None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:05.947073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:05.953299 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a7637-bbcc-4751-bc8c-980ee7dc4976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:05.953912 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a7637-bbcc-4751-bc8c-980ee7dc4976 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:05.954191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a7637-bbcc-4751-bc8c-980ee7dc4976 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:05.954464 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a7637-bbcc-4751-bc8c-980ee7dc4976 None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:05.955095 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a7637-bbcc-4751-bc8c-980ee7dc4976 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:05.955401 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a7637-bbcc-4751-bc8c-980ee7dc4976 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:05.956793 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a7637-bbcc-4751-bc8c-980ee7dc4976 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:05.966246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 517/1016] 10.222.0.59 () {56 vars in 1112 bytes} [Fri Aug 26 00:36:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:36:05.968333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9adfcfd-ec24-45be-9f58-4579bf0e9390 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:06.063125 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f5744-bdea-44cd-bb71-ab1d2ee644f3 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:06.063420 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f5744-bdea-44cd-bb71-ab1d2ee644f3 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:06.063693 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f5744-bdea-44cd-bb71-ab1d2ee644f3 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:06.065204 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 518/1017] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:36:06.069852 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718a587-fc93-4c0d-aa88-0ebb5310b855 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:06.070137 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718a587-fc93-4c0d-aa88-0ebb5310b855 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:06.070397 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9adfcfd-ec24-45be-9f58-4579bf0e9390 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:06.070523 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718a587-fc93-4c0d-aa88-0ebb5310b855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:06.088335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-d718a587-fc93-4c0d-aa88-0ebb5310b855 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:06.089537 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-d718a587-fc93-4c0d-aa88-0ebb5310b855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:06.125529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5233b2e-d13c-4881-9362-e5d3ca179114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:06.126338 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5233b2e-d13c-4881-9362-e5d3ca179114 None admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:06.126833 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5233b2e-d13c-4881-9362-e5d3ca179114 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:06.127272 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5233b2e-d13c-4881-9362-e5d3ca179114 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:06.128110 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5233b2e-d13c-4881-9362-e5d3ca179114 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:06.129694 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5233b2e-d13c-4881-9362-e5d3ca179114 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:06.159483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 519/1018] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:06 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 00:36:06.197502 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 500/1019] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:36:05 2022] POST /identity/v3/endpoints => generated 334 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:36:06.425294 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718a587-fc93-4c0d-aa88-0ebb5310b855 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:06.535173 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718a587-fc93-4c0d-aa88-0ebb5310b855 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:06.560635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:06.577129 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ae9bc-e65a-444b-9360-8bb14b010545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:06.579890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ae9bc-e65a-444b-9360-8bb14b010545 None admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:06.579890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ae9bc-e65a-444b-9360-8bb14b010545 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:06.579890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ae9bc-e65a-444b-9360-8bb14b010545 None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:06.579890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ae9bc-e65a-444b-9360-8bb14b010545 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:06.580335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ae9bc-e65a-444b-9360-8bb14b010545 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:06.689624 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 520/1020] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Aug 26 00:36:06 2022] POST /identity/v3/services => generated 242 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:36:07.501569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63ddec-6cff-4470-b4d9-dee3072c6c75 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:07.502073 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63ddec-6cff-4470-b4d9-dee3072c6c75 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:07.502181 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63ddec-6cff-4470-b4d9-dee3072c6c75 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:07.503571 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 501/1021] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:36:07.508543 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d91c2-1f83-4d84-9053-33a19c0b9b42 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:07.508922 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d91c2-1f83-4d84-9053-33a19c0b9b42 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:07.509090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d91c2-1f83-4d84-9053-33a19c0b9b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:07.526678 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-361d91c2-1f83-4d84-9053-33a19c0b9b42 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:07.527842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-361d91c2-1f83-4d84-9053-33a19c0b9b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:07.600504 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 521/1022] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:36:07.866457 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63ddec-6cff-4470-b4d9-dee3072c6c75 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:07.914928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:07.971757 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63ddec-6cff-4470-b4d9-dee3072c6c75 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:08.004493 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911cd6f-1c9a-4026-bd2a-971989ae7146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:08.005144 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911cd6f-1c9a-4026-bd2a-971989ae7146 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:08.005415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911cd6f-1c9a-4026-bd2a-971989ae7146 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:08.005686 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911cd6f-1c9a-4026-bd2a-971989ae7146 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:08.006783 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7911cd6f-1c9a-4026-bd2a-971989ae7146 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:08.008204 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7911cd6f-1c9a-4026-bd2a-971989ae7146 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:08.009994 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 502/1023] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:36:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:08.015467 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d91c2-1f83-4d84-9053-33a19c0b9b42 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:08.125988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d91c2-1f83-4d84-9053-33a19c0b9b42 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:08.137530 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:08.176157 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455b5c5-af9f-4ab9-9e0e-935d395c5f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:08.176808 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455b5c5-af9f-4ab9-9e0e-935d395c5f9d None admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:08.177090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455b5c5-af9f-4ab9-9e0e-935d395c5f9d None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:08.177364 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455b5c5-af9f-4ab9-9e0e-935d395c5f9d None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:08.178015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f455b5c5-af9f-4ab9-9e0e-935d395c5f9d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:08.179515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f455b5c5-af9f-4ab9-9e0e-935d395c5f9d None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:08.230513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50afd8-e8bd-4689-a382-9f914c61d20a None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:08.230803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50afd8-e8bd-4689-a382-9f914c61d20a None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:08.231080 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50afd8-e8bd-4689-a382-9f914c61d20a None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:08.233213 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 503/1024] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:36:08.237671 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e31890-bfb5-443c-a0ce-987d4a286c16 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:08.237953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e31890-bfb5-443c-a0ce-987d4a286c16 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:08.238228 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e31890-bfb5-443c-a0ce-987d4a286c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:08.255810 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-a0e31890-bfb5-443c-a0ce-987d4a286c16 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:08.257030 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-a0e31890-bfb5-443c-a0ce-987d4a286c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:08.292509 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 522/1025] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Aug 26 00:36:08 2022] POST /identity/v3/users => generated 285 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:36:08.330470 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 504/1026] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 00:36:08.601709 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455b5c5-af9f-4ab9-9e0e-935d395c5f9d None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:08.617631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181416e2-93ee-413f-bf3d-23656e397811 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:08.618116 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181416e2-93ee-413f-bf3d-23656e397811 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:08.618207 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181416e2-93ee-413f-bf3d-23656e397811 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:08.619720 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 505/1027] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:36:08.625150 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310b12a-e8f2-471e-b487-9874086fa971 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:08.625443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310b12a-e8f2-471e-b487-9874086fa971 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:08.625721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310b12a-e8f2-471e-b487-9874086fa971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:08.643573 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-e310b12a-e8f2-471e-b487-9874086fa971 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:08.644801 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-e310b12a-e8f2-471e-b487-9874086fa971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:08.682748 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:08.710732 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455b5c5-af9f-4ab9-9e0e-935d395c5f9d None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:08.716255 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 506/1028] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 00:36:08.748757 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43379bb0-0367-4072-b30b-4746be4b1da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:08.749671 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43379bb0-0367-4072-b30b-4746be4b1da5 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:08.749954 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43379bb0-0367-4072-b30b-4746be4b1da5 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:08.750230 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43379bb0-0367-4072-b30b-4746be4b1da5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:08.761209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43379bb0-0367-4072-b30b-4746be4b1da5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:36:08.765935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-43379bb0-0367-4072-b30b-4746be4b1da5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 00:36:08.765935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:08.765935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:08.765935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:08.765935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:08.765935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:08.765935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:08.765935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:08.765935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:08.765935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:08.765935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:08.765935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:08.765935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:36:08.765935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:36:08.765935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:36:08.765935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:36:08.766890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:36:08.766890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:36:08.766890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:36:08.766890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:36:08.766890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:08.766890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:08.766890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:08.766890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:08.766890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:08.766890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:08.766890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:08.766890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:08.766890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:08.766890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:08.766890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:08.766890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:08.767893 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:08.767893 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:08.767893 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:08.767893 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:08.767893 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:36:08.767893 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:36:08.767893 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:36:08.767893 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:36:08.767893 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:36:08.767893 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:36:08.767893 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 00:36:08.767893 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:36:08.769268 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 523/1029] 10.222.0.59 () {56 vars in 1120 bytes} [Fri Aug 26 00:36:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:36:08.772946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310b12a-e8f2-471e-b487-9874086fa971 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:08.874678 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310b12a-e8f2-471e-b487-9874086fa971 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:08.914080 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517b2be-e494-4511-8085-ee42595cbbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:08.914689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517b2be-e494-4511-8085-ee42595cbbbf None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:08.914971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517b2be-e494-4511-8085-ee42595cbbbf None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:08.915242 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517b2be-e494-4511-8085-ee42595cbbbf None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:08.915901 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517b2be-e494-4511-8085-ee42595cbbbf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:08.916204 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517b2be-e494-4511-8085-ee42595cbbbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:08.917610 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517b2be-e494-4511-8085-ee42595cbbbf None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:08.925973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 507/1030] 10.222.0.59 () {56 vars in 1115 bytes} [Fri Aug 26 00:36:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:08.931729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43379bb0-0367-4072-b30b-4746be4b1da5 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:08.957166 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:08.995053 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517b2be-e494-4511-8085-ee42595cbbbf None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:09.031057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:09.034560 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43379bb0-0367-4072-b30b-4746be4b1da5 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:09.070475 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcb58cd-42ac-4e19-9b82-7a95e9099141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:09.071107 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb58cd-42ac-4e19-9b82-7a95e9099141 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:09.071394 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb58cd-42ac-4e19-9b82-7a95e9099141 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:09.071668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb58cd-42ac-4e19-9b82-7a95e9099141 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:09.082548 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcb58cd-42ac-4e19-9b82-7a95e9099141 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:09.083995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcb58cd-42ac-4e19-9b82-7a95e9099141 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:09.099254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-efcb58cd-42ac-4e19-9b82-7a95e9099141 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:36:09.099254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:09.099254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:09.099254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:09.099254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:09.099254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:09.099254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:09.099254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:09.099254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:09.099254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:09.099254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:09.099254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:09.099254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:36:09.099254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:36:09.099254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:36:09.099254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:36:09.100259 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:09.100259 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:09.100259 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:09.100259 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:09.100259 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:09.100259 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:09.100259 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:09.100259 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:09.100259 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:09.100259 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:09.100259 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:09.100259 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:09.100259 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:09.100259 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:09.100259 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:09.100259 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:09.100259 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:36:09.101242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:36:09.101242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:36:09.101242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:36:09.104838 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 524/1031] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:36:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:36:09.106466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcb58cd-42ac-4e19-9b82-7a95e9099141 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:09.111438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517b2be-e494-4511-8085-ee42595cbbbf None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:09.147415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf279c-cea0-4f58-977a-f1b137109455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:09.148177 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf279c-cea0-4f58-977a-f1b137109455 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:09.148460 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf279c-cea0-4f58-977a-f1b137109455 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:09.148758 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf279c-cea0-4f58-977a-f1b137109455 None admin] PATH_INFO: `/v3/services/image` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:09.149463 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf279c-cea0-4f58-977a-f1b137109455 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:09.150946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf279c-cea0-4f58-977a-f1b137109455 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:09.161029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-afcf279c-cea0-4f58-977a-f1b137109455 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 00:36:09.161029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:09.161029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 00:36:09.161029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 00:36:09.161029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 00:36:09.161029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 00:36:09.161029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 00:36:09.161029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:36:09.161029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 00:36:09.161029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:36:09.161029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:36:09.161029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:36:09.161029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:09.161029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:09.161029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:09.161029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:09.161029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:09.162067 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:09.162067 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:09.162067 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:09.162067 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:09.162067 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:09.162067 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:09.162067 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 00:36:09.162067 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 00:36:09.162067 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 00:36:09.162067 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 00:36:09.162067 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:09.162067 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:09.162067 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:09.162067 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:09.162067 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:09.162067 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:09.163092 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:09.163092 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:09.163092 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:09.163092 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:09.163092 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:09.163092 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:09.163092 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:09.163092 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:09.163092 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:09.163092 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:09.163092 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 00:36:09.163092 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:36:09.163092 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 00:36:09.163092 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:36:09.163092 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 508/1032] 10.222.0.59 () {56 vars in 1105 bytes} [Fri Aug 26 00:36:08 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:36:09.168187 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf279c-cea0-4f58-977a-f1b137109455 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:09.194423 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:09.220442 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcb58cd-42ac-4e19-9b82-7a95e9099141 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:09.257401 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246314e-e3e3-42ca-a8fd-68bb92d77e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:09.258103 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246314e-e3e3-42ca-a8fd-68bb92d77e02 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:09.258386 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246314e-e3e3-42ca-a8fd-68bb92d77e02 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:09.258663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246314e-e3e3-42ca-a8fd-68bb92d77e02 None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:09.259289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246314e-e3e3-42ca-a8fd-68bb92d77e02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:09.259589 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246314e-e3e3-42ca-a8fd-68bb92d77e02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:09.260996 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246314e-e3e3-42ca-a8fd-68bb92d77e02 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:09.273799 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 525/1033] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:36:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:36:09.275410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:09.280072 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246314e-e3e3-42ca-a8fd-68bb92d77e02 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:09.282490 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf279c-cea0-4f58-977a-f1b137109455 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:09.318639 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002caa0-d44a-4a1f-ad4a-b1d773f9a8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:09.319251 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002caa0-d44a-4a1f-ad4a-b1d773f9a8b0 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:09.319529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002caa0-d44a-4a1f-ad4a-b1d773f9a8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:09.319801 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002caa0-d44a-4a1f-ad4a-b1d773f9a8b0 None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:09.320431 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3002caa0-d44a-4a1f-ad4a-b1d773f9a8b0 None admin] RBAC: Adding query filter params (name=image) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:09.320760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3002caa0-d44a-4a1f-ad4a-b1d773f9a8b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:09.322125 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3002caa0-d44a-4a1f-ad4a-b1d773f9a8b0 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:09.328509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 509/1034] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:36:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:36:09.333173 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002caa0-d44a-4a1f-ad4a-b1d773f9a8b0 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:09.393077 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246314e-e3e3-42ca-a8fd-68bb92d77e02 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:09.433859 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b4b92-ea42-476e-a81c-84ed7a20a169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:09.434878 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b4b92-ea42-476e-a81c-84ed7a20a169 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:09.435386 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b4b92-ea42-476e-a81c-84ed7a20a169 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:09.436245 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b4b92-ea42-476e-a81c-84ed7a20a169 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:09.438627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:09.444625 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1b4b92-ea42-476e-a81c-84ed7a20a169 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:09.446164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1b4b92-ea42-476e-a81c-84ed7a20a169 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:09.453204 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002caa0-d44a-4a1f-ad4a-b1d773f9a8b0 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:09.458266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-2f1b4b92-ea42-476e-a81c-84ed7a20a169 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 00:36:09.458266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:09.458266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:09.458266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:09.458266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:09.458266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:09.458266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:09.458266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:09.458266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:09.458266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:09.458266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:09.458266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:09.458266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:36:09.458266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:36:09.458266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:36:09.458266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:36:09.459191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:09.459191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:09.459191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:36:09.459191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:36:09.459191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:36:09.459191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:36:09.459191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:09.459191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:09.459191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:09.459191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:09.459191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:09.459191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:09.459191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:09.459191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:09.459191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:09.459191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:09.459191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:09.459191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:09.460150 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:09.460150 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:09.460150 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:36:09.460150 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:36:09.460150 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:36:09.460150 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:36:09.460150 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:36:09.460150 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:36:09.460150 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 00:36:09.460150 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:36:09.464264 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 526/1035] 10.222.0.59 () {56 vars in 1093 bytes} [Fri Aug 26 00:36:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:36:09.465146 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b4b92-ea42-476e-a81c-84ed7a20a169 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:09.489543 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e0654-c708-4147-b795-e7f88f20c37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:09.490388 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e0654-c708-4147-b795-e7f88f20c37d None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:09.490683 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e0654-c708-4147-b795-e7f88f20c37d None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:09.490960 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e0654-c708-4147-b795-e7f88f20c37d None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:09.491583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e0654-c708-4147-b795-e7f88f20c37d None admin] RBAC: Adding query filter params (type=image) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:09.492062 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e0654-c708-4147-b795-e7f88f20c37d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:09.493458 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e0654-c708-4147-b795-e7f88f20c37d None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:09.499148 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 510/1036] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:36:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:36:09.505349 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e0654-c708-4147-b795-e7f88f20c37d None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:09.532007 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:09.574034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b4b92-ea42-476e-a81c-84ed7a20a169 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:09.607376 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e0654-c708-4147-b795-e7f88f20c37d None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:09.609940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9facbcae-4014-42c0-a64b-ca3f529f3324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:09.610624 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9facbcae-4014-42c0-a64b-ca3f529f3324 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:09.610966 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9facbcae-4014-42c0-a64b-ca3f529f3324 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:09.611304 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9facbcae-4014-42c0-a64b-ca3f529f3324 None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:09.612109 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9facbcae-4014-42c0-a64b-ca3f529f3324 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:09.612495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9facbcae-4014-42c0-a64b-ca3f529f3324 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:09.614000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9facbcae-4014-42c0-a64b-ca3f529f3324 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:09.633757 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 527/1037] 10.222.0.59 () {56 vars in 1133 bytes} [Fri Aug 26 00:36:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:09.639577 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9facbcae-4014-42c0-a64b-ca3f529f3324 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:09.645727 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222d66a-88d3-49fb-9b46-e71213d4ee89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:09.646335 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222d66a-88d3-49fb-9b46-e71213d4ee89 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:09.646620 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222d66a-88d3-49fb-9b46-e71213d4ee89 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:09.646895 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222d66a-88d3-49fb-9b46-e71213d4ee89 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:09.647585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222d66a-88d3-49fb-9b46-e71213d4ee89 None admin] RBAC: Adding query filter params (interface=public, service_id=c3d7e09787594c8a9f9bb8bfd0428561, region_id=RegionOne) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:09.648037 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222d66a-88d3-49fb-9b46-e71213d4ee89 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:09.649443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222d66a-88d3-49fb-9b46-e71213d4ee89 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:09.655046 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 511/1038] 10.222.0.59 () {56 vars in 1251 bytes} [Fri Aug 26 00:36:09 2022] GET /identity/v3/endpoints?service_id=c3d7e09787594c8a9f9bb8bfd0428561&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:09.660413 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222d66a-88d3-49fb-9b46-e71213d4ee89 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:09.698066 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:09.750886 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9facbcae-4014-42c0-a64b-ca3f529f3324 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:09.754678 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:09.767911 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222d66a-88d3-49fb-9b46-e71213d4ee89 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:09.788291 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29138fba-f3de-4982-8285-9b24e1f3f24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:09.788873 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29138fba-f3de-4982-8285-9b24e1f3f24b None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:09.789153 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29138fba-f3de-4982-8285-9b24e1f3f24b None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:09.789846 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29138fba-f3de-4982-8285-9b24e1f3f24b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:09.805681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5322d-f4c5-4d91-9ed3-ae77622b8de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:09.806402 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5322d-f4c5-4d91-9ed3-ae77622b8de8 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:09.806746 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5322d-f4c5-4d91-9ed3-ae77622b8de8 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:09.807082 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5322d-f4c5-4d91-9ed3-ae77622b8de8 None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:09.807750 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5322d-f4c5-4d91-9ed3-ae77622b8de8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:09.809135 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5322d-f4c5-4d91-9ed3-ae77622b8de8 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:09.824505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 512/1039] 10.222.0.59 () {56 vars in 1087 bytes} [Fri Aug 26 00:36:09 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:36:09.877854 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29138fba-f3de-4982-8285-9b24e1f3f24b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:09.879334 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29138fba-f3de-4982-8285-9b24e1f3f24b None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:09.936895 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:09.988070 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 528/1040] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:36:09 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 00:36:09.994158 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5322d-f4c5-4d91-9ed3-ae77622b8de8 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:10.098829 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5322d-f4c5-4d91-9ed3-ae77622b8de8 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:10.133822 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1f307-c039-431e-80ed-1600eb194d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:10.134516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1f307-c039-431e-80ed-1600eb194d34 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:10.134800 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1f307-c039-431e-80ed-1600eb194d34 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:10.135072 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1f307-c039-431e-80ed-1600eb194d34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:10.145965 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1f307-c039-431e-80ed-1600eb194d34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:10.147420 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1f307-c039-431e-80ed-1600eb194d34 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:10.162943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-9be1f307-c039-431e-80ed-1600eb194d34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:36:10.162943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:10.162943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:10.162943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:10.162943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:10.162943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:10.162943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:10.162943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:10.162943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:10.162943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:10.162943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:10.162943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:10.162943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:36:10.162943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:36:10.162943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:36:10.162943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:36:10.164018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:10.164018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:10.164018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:10.164018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:10.164018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:10.164018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:10.164018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:10.164018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:10.164018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:10.164018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:10.164018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:10.164018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:10.164018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:10.164018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:10.164018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:10.164018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:10.164018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:36:10.164987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:36:10.164987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:36:10.164987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:36:10.164987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 513/1041] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:36:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:36:10.170529 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29138fba-f3de-4982-8285-9b24e1f3f24b None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:10.255480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8970c-3ef9-4cc4-9c7f-c75226aeadb8 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:10.255845 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8970c-3ef9-4cc4-9c7f-c75226aeadb8 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:10.256036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8970c-3ef9-4cc4-9c7f-c75226aeadb8 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:10.257621 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 514/1042] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:36:10.262245 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707179b1-400f-42b7-ace1-ae8d21e45045 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:10.262529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707179b1-400f-42b7-ace1-ae8d21e45045 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:10.262794 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707179b1-400f-42b7-ace1-ae8d21e45045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:10.270562 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:10.273406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29138fba-f3de-4982-8285-9b24e1f3f24b None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:10.281652 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-707179b1-400f-42b7-ace1-ae8d21e45045 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:10.282826 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-707179b1-400f-42b7-ace1-ae8d21e45045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:10.311260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee58303-2e4e-40af-81e9-923e7a44d64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:10.311873 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee58303-2e4e-40af-81e9-923e7a44d64f None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:10.312150 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee58303-2e4e-40af-81e9-923e7a44d64f None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:10.312422 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee58303-2e4e-40af-81e9-923e7a44d64f None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:10.313097 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee58303-2e4e-40af-81e9-923e7a44d64f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:10.313398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee58303-2e4e-40af-81e9-923e7a44d64f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:10.314754 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee58303-2e4e-40af-81e9-923e7a44d64f None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:10.325994 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 529/1043] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:10.331487 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee58303-2e4e-40af-81e9-923e7a44d64f None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:10.347041 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:10.361096 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 515/1044] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 00:36:10.442916 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee58303-2e4e-40af-81e9-923e7a44d64f None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:10.478479 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504f8fc-c802-444d-9cc5-38a9840e2d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:10.479185 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504f8fc-c802-444d-9cc5-38a9840e2d95 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:10.479474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504f8fc-c802-444d-9cc5-38a9840e2d95 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:10.479751 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504f8fc-c802-444d-9cc5-38a9840e2d95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:10.487088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504f8fc-c802-444d-9cc5-38a9840e2d95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:10.488629 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504f8fc-c802-444d-9cc5-38a9840e2d95 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:10.500282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-3504f8fc-c802-444d-9cc5-38a9840e2d95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:36:10.500282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:10.500282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:10.500282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:10.500282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:10.500282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:10.500282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:10.500282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:10.500282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:10.500282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:10.500282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:10.500282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:10.500282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:36:10.500282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:36:10.500282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:36:10.501410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:36:10.501410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:10.501410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:10.501410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:10.501410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:10.501410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:10.501410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:10.501410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:10.501410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:10.501410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:10.501410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:10.501410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:10.501410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:10.501410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:10.501410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:10.501410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:10.501410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:10.509601 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:36:10.509601 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:36:10.509601 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:36:10.509601 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:36:10.509601 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:36:10.509601 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:36:10.509601 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:36:10.509601 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:36:10.509601 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 530/1045] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:36:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:36:10.514285 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707179b1-400f-42b7-ace1-ae8d21e45045 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:10.624385 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707179b1-400f-42b7-ace1-ae8d21e45045 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:10.627479 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504f8fc-c802-444d-9cc5-38a9840e2d95 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:10.661576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb353a4-8d03-4eb8-aae8-01191a2fee6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:10.662183 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb353a4-8d03-4eb8-aae8-01191a2fee6c None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:10.662471 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb353a4-8d03-4eb8-aae8-01191a2fee6c None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:10.662744 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb353a4-8d03-4eb8-aae8-01191a2fee6c None admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:10.663438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb353a4-8d03-4eb8-aae8-01191a2fee6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:10.663809 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb353a4-8d03-4eb8-aae8-01191a2fee6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:10.665267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb353a4-8d03-4eb8-aae8-01191a2fee6c None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:10.667054 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:10.680650 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 516/1046] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:36:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:10.684755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb353a4-8d03-4eb8-aae8-01191a2fee6c None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:10.704180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:10.736857 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504f8fc-c802-444d-9cc5-38a9840e2d95 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:10.773025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877da417-2024-4c4a-b8bd-7735e6f29cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:10.773700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877da417-2024-4c4a-b8bd-7735e6f29cbb None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:10.773984 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877da417-2024-4c4a-b8bd-7735e6f29cbb None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:10.774534 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877da417-2024-4c4a-b8bd-7735e6f29cbb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:10.785396 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877da417-2024-4c4a-b8bd-7735e6f29cbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:36:10.790024 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb353a4-8d03-4eb8-aae8-01191a2fee6c None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:10.790474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-877da417-2024-4c4a-b8bd-7735e6f29cbb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:36:10.790474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:10.790474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:10.790474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:10.790474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:10.790474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:10.790474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:10.790474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:10.790474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:10.790474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:10.790474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:10.790474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:10.790474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:36:10.790474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:36:10.790474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:36:10.790474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:36:10.791488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:36:10.791488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:36:10.791488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:36:10.791488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:36:10.791488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:10.791488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:10.791488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:10.791488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:10.791488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:10.791488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:10.791488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:10.791488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:10.791488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:10.791488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:10.791488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:10.791488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:10.792369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:10.792369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:10.792369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:10.792369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:10.792369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:36:10.792369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:36:10.792369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:36:10.792369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:36:10.792369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:36:10.792369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:36:10.792369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:36:10.792369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:36:10.792369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 531/1047] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:36:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:36:10.798015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877da417-2024-4c4a-b8bd-7735e6f29cbb None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:10.829691 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06d124-5a53-49a4-9c63-021011a9904a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:10.830307 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06d124-5a53-49a4-9c63-021011a9904a None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:10.830583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06d124-5a53-49a4-9c63-021011a9904a None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:10.830849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06d124-5a53-49a4-9c63-021011a9904a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:10.831572 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06d124-5a53-49a4-9c63-021011a9904a None admin] RBAC: Adding query filter params (role.id=91940e122e2f46e3818c5f11f4fca9e9, scope.project.id=fde7ac3ed472433ab3bba56e5e091510, user.id=0602593263ca4b05a657d5b7697f932f) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:10.831878 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06d124-5a53-49a4-9c63-021011a9904a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:10.833340 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06d124-5a53-49a4-9c63-021011a9904a None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:10.839746 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 517/1048] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:36:10 2022] GET /identity/v3/role_assignments?user.id=0602593263ca4b05a657d5b7697f932f&scope.project.id=fde7ac3ed472433ab3bba56e5e091510&role.id=91940e122e2f46e3818c5f11f4fca9e9 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:36:10.914154 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877da417-2024-4c4a-b8bd-7735e6f29cbb None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:10.947539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:10.953864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc24396-1750-46e5-9119-b9de44eb05a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:10.954496 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc24396-1750-46e5-9119-b9de44eb05a4 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:10.954775 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc24396-1750-46e5-9119-b9de44eb05a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:10.955044 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc24396-1750-46e5-9119-b9de44eb05a4 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:10.955721 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc24396-1750-46e5-9119-b9de44eb05a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:10.956020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc24396-1750-46e5-9119-b9de44eb05a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:10.957541 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc24396-1750-46e5-9119-b9de44eb05a4 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:10.972403 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 532/1049] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:36:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:36:10.973991 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06d124-5a53-49a4-9c63-021011a9904a None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:11.037347 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:11.080139 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06d124-5a53-49a4-9c63-021011a9904a None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:11.114122 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5e5ce-3b9f-43ea-9ac1-143841523c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:11.114742 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5e5ce-3b9f-43ea-9ac1-143841523c60 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:11.115015 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5e5ce-3b9f-43ea-9ac1-143841523c60 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:11.115282 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5e5ce-3b9f-43ea-9ac1-143841523c60 None admin] PATH_INFO: `/v3/users/3927365b4b08436ba8b5e4f71690482d` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:11.128768 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f5e5ce-3b9f-43ea-9ac1-143841523c60 None admin] RBAC: Authorizing `identity:get_user(user_id=3927365b4b08436ba8b5e4f71690482d)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:11.130206 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f5e5ce-3b9f-43ea-9ac1-143841523c60 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:11.131988 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 518/1050] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:36:10 2022] GET /identity/v3/users/3927365b4b08436ba8b5e4f71690482d => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:11.137828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc24396-1750-46e5-9119-b9de44eb05a4 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:11.240770 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc24396-1750-46e5-9119-b9de44eb05a4 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:11.275591 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae813f-ba8d-431b-b325-8e5655d05efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:11.276201 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae813f-ba8d-431b-b325-8e5655d05efe None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:11.276494 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae813f-ba8d-431b-b325-8e5655d05efe None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:11.276797 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae813f-ba8d-431b-b325-8e5655d05efe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:11.336986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:11.345284 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae813f-ba8d-431b-b325-8e5655d05efe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:11.346743 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae813f-ba8d-431b-b325-8e5655d05efe None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:11.459476 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 533/1051] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:36:11 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 00:36:11.461306 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5e5ce-3b9f-43ea-9ac1-143841523c60 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:11.561455 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5e5ce-3b9f-43ea-9ac1-143841523c60 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:11.578074 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:11.595894 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7effb117-dcdf-4323-b6ca-4f9c72490b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:11.596555 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effb117-dcdf-4323-b6ca-4f9c72490b3a None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:11.596833 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effb117-dcdf-4323-b6ca-4f9c72490b3a None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:11.597100 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effb117-dcdf-4323-b6ca-4f9c72490b3a None admin] PATH_INFO: `/v3/projects/abf5cf909af24d2281230165a0bd8526` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:11.598234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7effb117-dcdf-4323-b6ca-4f9c72490b3a None admin] RBAC: Authorizing `identity:get_project(project_id=abf5cf909af24d2281230165a0bd8526)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:11.599688 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7effb117-dcdf-4323-b6ca-4f9c72490b3a None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:11.606427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 519/1052] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:36:11 2022] GET /identity/v3/projects/abf5cf909af24d2281230165a0bd8526 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:11.607664 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae813f-ba8d-431b-b325-8e5655d05efe None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:11.702712 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:11.710341 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae813f-ba8d-431b-b325-8e5655d05efe None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:11.742500 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84567396-94c6-4c05-b3eb-43e359081a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:11.743168 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84567396-94c6-4c05-b3eb-43e359081a18 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:11.743452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84567396-94c6-4c05-b3eb-43e359081a18 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:11.743726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84567396-94c6-4c05-b3eb-43e359081a18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:11.744444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84567396-94c6-4c05-b3eb-43e359081a18 None admin] RBAC: Adding query filter params (role.id=1a33e3c8cb8347feb9498595886cc621, scope.project.id=abf5cf909af24d2281230165a0bd8526, user.id=3927365b4b08436ba8b5e4f71690482d) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:11.744795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84567396-94c6-4c05-b3eb-43e359081a18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:11.746204 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84567396-94c6-4c05-b3eb-43e359081a18 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:11.752734 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 534/1053] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:36:11 2022] GET /identity/v3/role_assignments?user.id=3927365b4b08436ba8b5e4f71690482d&scope.project.id=abf5cf909af24d2281230165a0bd8526&role.id=1a33e3c8cb8347feb9498595886cc621 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:11.793365 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b74fe-47b7-4054-9a89-c0661dab0788 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:11.793888 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b74fe-47b7-4054-9a89-c0661dab0788 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:11.793967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b74fe-47b7-4054-9a89-c0661dab0788 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:11.795596 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 520/1054] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:36:11.801515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3f0c9-1f0a-416f-a2f7-3a0dedf8adf3 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:11.801835 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3f0c9-1f0a-416f-a2f7-3a0dedf8adf3 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:11.802074 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3f0c9-1f0a-416f-a2f7-3a0dedf8adf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:11.822040 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-80d3f0c9-1f0a-416f-a2f7-3a0dedf8adf3 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:11.823180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-80d3f0c9-1f0a-416f-a2f7-3a0dedf8adf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:11.906667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 535/1055] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:36:12.176082 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b74fe-47b7-4054-9a89-c0661dab0788 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:12.291634 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b74fe-47b7-4054-9a89-c0661dab0788 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:12.324772 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3206f44b-9782-44f5-9dc1-b363bcf292c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:12.325403 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206f44b-9782-44f5-9dc1-b363bcf292c8 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:12.325680 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206f44b-9782-44f5-9dc1-b363bcf292c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:12.325955 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206f44b-9782-44f5-9dc1-b363bcf292c8 None admin] PATH_INFO: `/v3/services/image` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:12.327057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3206f44b-9782-44f5-9dc1-b363bcf292c8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:12.328412 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3206f44b-9782-44f5-9dc1-b363bcf292c8 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:12.336947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-3206f44b-9782-44f5-9dc1-b363bcf292c8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 00:36:12.336947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:12.336947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 00:36:12.336947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 00:36:12.336947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 00:36:12.336947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 00:36:12.336947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 00:36:12.336947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:36:12.336947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 00:36:12.336947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:36:12.336947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:36:12.336947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:36:12.336947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:12.336947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:12.336947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:12.336947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:12.336947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:12.337944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:12.337944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:12.337944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:12.337944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:12.337944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:12.337944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:12.337944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 00:36:12.337944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 00:36:12.337944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 00:36:12.337944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 00:36:12.337944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:12.337944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:12.337944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:12.337944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:12.337944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:12.337944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:12.339459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:12.339459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:12.339459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:12.339459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:12.339459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:12.339459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:12.339459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:12.339459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:12.339459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:12.339459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:12.339459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 00:36:12.339459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:36:12.339459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 00:36:12.339459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:36:12.339459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 521/1056] 10.222.0.59 () {56 vars in 1105 bytes} [Fri Aug 26 00:36:12 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:36:12.344429 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3f0c9-1f0a-416f-a2f7-3a0dedf8adf3 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:12.442568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:12.455375 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3f0c9-1f0a-416f-a2f7-3a0dedf8adf3 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:12.491305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816e2b6-43b1-4d38-ba5b-8566695d5384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:12.492314 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816e2b6-43b1-4d38-ba5b-8566695d5384 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:12.492850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816e2b6-43b1-4d38-ba5b-8566695d5384 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:12.493344 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816e2b6-43b1-4d38-ba5b-8566695d5384 None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:12.494168 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816e2b6-43b1-4d38-ba5b-8566695d5384 None admin] RBAC: Adding query filter params (name=image) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:12.494667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816e2b6-43b1-4d38-ba5b-8566695d5384 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:12.496223 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816e2b6-43b1-4d38-ba5b-8566695d5384 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:12.502462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 536/1057] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:36:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:12.507909 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3206f44b-9782-44f5-9dc1-b363bcf292c8 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:12.560451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:12.613197 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3206f44b-9782-44f5-9dc1-b363bcf292c8 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:12.646393 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63543df6-9e03-4e6d-9182-f1dc479512bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:12.646998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63543df6-9e03-4e6d-9182-f1dc479512bb None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:12.647446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63543df6-9e03-4e6d-9182-f1dc479512bb None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:12.647722 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63543df6-9e03-4e6d-9182-f1dc479512bb None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:12.648344 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63543df6-9e03-4e6d-9182-f1dc479512bb None admin] RBAC: Adding query filter params (type=image) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:12.648667 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63543df6-9e03-4e6d-9182-f1dc479512bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:12.650018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63543df6-9e03-4e6d-9182-f1dc479512bb None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:12.660209 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 522/1058] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:36:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:12.661456 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816e2b6-43b1-4d38-ba5b-8566695d5384 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:12.763241 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816e2b6-43b1-4d38-ba5b-8566695d5384 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:12.786212 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61010d7-d5bf-47e0-b895-183e3c8b4fa4 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:12.786513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61010d7-d5bf-47e0-b895-183e3c8b4fa4 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:12.786779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61010d7-d5bf-47e0-b895-183e3c8b4fa4 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:12.788353 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 523/1059] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:36:12.793251 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49442f-99cc-40ca-8baa-bc0f31e59175 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:12.793546 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49442f-99cc-40ca-8baa-bc0f31e59175 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:12.794084 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49442f-99cc-40ca-8baa-bc0f31e59175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:12.799003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe80a6-f319-4a72-b228-096e2cabb0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:12.799616 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe80a6-f319-4a72-b228-096e2cabb0c7 None admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:12.800019 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe80a6-f319-4a72-b228-096e2cabb0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:12.800295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe80a6-f319-4a72-b228-096e2cabb0c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:12.800988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe80a6-f319-4a72-b228-096e2cabb0c7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:12.802356 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe80a6-f319-4a72-b228-096e2cabb0c7 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:12.811917 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-2e49442f-99cc-40ca-8baa-bc0f31e59175 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:12.813386 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-2e49442f-99cc-40ca-8baa-bc0f31e59175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:12.881184 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 524/1060] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 00:36:12.892550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 537/1061] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:36:12 2022] POST /identity/v3/endpoints => generated 327 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:36:13.155311 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49442f-99cc-40ca-8baa-bc0f31e59175 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:13.199230 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:13.255890 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49442f-99cc-40ca-8baa-bc0f31e59175 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:13.296807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b0370-81e4-478b-bfda-2a9fc4113fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:13.297450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b0370-81e4-478b-bfda-2a9fc4113fad None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:13.297724 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b0370-81e4-478b-bfda-2a9fc4113fad None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:13.297989 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b0370-81e4-478b-bfda-2a9fc4113fad None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:13.307971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b0370-81e4-478b-bfda-2a9fc4113fad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:36:13.312699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-7e6b0370-81e4-478b-bfda-2a9fc4113fad None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 00:36:13.312699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:13.312699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:13.312699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:13.312699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:13.312699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:13.312699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:13.312699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:13.312699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:13.312699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:13.312699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:13.312699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:13.312699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:36:13.312699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:36:13.312699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:36:13.312699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:36:13.313629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:36:13.313629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:36:13.313629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:36:13.313629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:36:13.313629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:13.313629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:13.313629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:13.313629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:13.313629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:13.313629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:13.313629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:13.313629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:13.313629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:13.313629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:13.313629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:13.313629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:13.317279 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:13.317279 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:13.317279 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:13.317279 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:13.317279 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:36:13.317279 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:36:13.317279 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:36:13.317279 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:36:13.317279 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:36:13.317279 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:36:13.317279 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 00:36:13.317279 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:36:13.317279 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 525/1062] 10.222.0.59 () {56 vars in 1120 bytes} [Fri Aug 26 00:36:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:36:13.320213 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe80a6-f319-4a72-b228-096e2cabb0c7 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:13.331854 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:13.424593 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe80a6-f319-4a72-b228-096e2cabb0c7 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:13.461833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67f212-e134-4b9d-9131-366d4a8ccde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:13.462440 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67f212-e134-4b9d-9131-366d4a8ccde5 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:13.462714 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67f212-e134-4b9d-9131-366d4a8ccde5 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:13.463009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67f212-e134-4b9d-9131-366d4a8ccde5 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:13.463664 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67f212-e134-4b9d-9131-366d4a8ccde5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:13.463964 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67f212-e134-4b9d-9131-366d4a8ccde5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:13.465390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67f212-e134-4b9d-9131-366d4a8ccde5 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:13.479142 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 538/1063] 10.222.0.59 () {56 vars in 1115 bytes} [Fri Aug 26 00:36:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:13.480959 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b0370-81e4-478b-bfda-2a9fc4113fad None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:13.582845 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:13.586663 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b0370-81e4-478b-bfda-2a9fc4113fad None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:13.620634 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e25c8e3-c781-4a77-8a95-df5e7647c207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:13.621268 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25c8e3-c781-4a77-8a95-df5e7647c207 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:13.621547 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25c8e3-c781-4a77-8a95-df5e7647c207 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:13.621817 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25c8e3-c781-4a77-8a95-df5e7647c207 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:13.632042 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e25c8e3-c781-4a77-8a95-df5e7647c207 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:13.633516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e25c8e3-c781-4a77-8a95-df5e7647c207 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:13.648541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-5e25c8e3-c781-4a77-8a95-df5e7647c207 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:36:13.648541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:13.648541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:13.648541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:13.648541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:13.648541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:13.648541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:13.648541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:13.648541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:13.648541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:13.648541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:13.648541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:13.648541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:36:13.648541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:36:13.648541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:36:13.648541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:36:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:13.649545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:36:13.655924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:36:13.655924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:36:13.655924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:36:13.655924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 526/1064] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:36:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:36:13.658139 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67f212-e134-4b9d-9131-366d4a8ccde5 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:13.684951 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ed85b-98bc-49d9-a3c2-0def8031773d None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:13.685240 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ed85b-98bc-49d9-a3c2-0def8031773d None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:13.685510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ed85b-98bc-49d9-a3c2-0def8031773d None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:13.687029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 527/1065] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:36:13.691867 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de771b-b974-4c54-9a91-cce20dfc6b71 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:13.692148 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de771b-b974-4c54-9a91-cce20dfc6b71 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:13.692419 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de771b-b974-4c54-9a91-cce20dfc6b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:13.712354 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-c8de771b-b974-4c54-9a91-cce20dfc6b71 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:13.713559 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-c8de771b-b974-4c54-9a91-cce20dfc6b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:13.730856 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:13.762630 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67f212-e134-4b9d-9131-366d4a8ccde5 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:13.785968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 528/1066] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:36:13.800924 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36e228-41eb-4170-bdc8-c3869fac4a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:13.801539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36e228-41eb-4170-bdc8-c3869fac4a48 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:13.801833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36e228-41eb-4170-bdc8-c3869fac4a48 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:13.802095 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36e228-41eb-4170-bdc8-c3869fac4a48 None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:13.802766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36e228-41eb-4170-bdc8-c3869fac4a48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:13.803102 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36e228-41eb-4170-bdc8-c3869fac4a48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:13.804447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36e228-41eb-4170-bdc8-c3869fac4a48 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:13.816607 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 539/1067] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:13.822010 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de771b-b974-4c54-9a91-cce20dfc6b71 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:13.903120 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:13.922244 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de771b-b974-4c54-9a91-cce20dfc6b71 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:13.959020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e81b2d-7a96-41c2-8408-6ad889c3004e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:13.959570 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e81b2d-7a96-41c2-8408-6ad889c3004e None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:13.959858 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e81b2d-7a96-41c2-8408-6ad889c3004e None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:13.960132 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e81b2d-7a96-41c2-8408-6ad889c3004e None admin] PATH_INFO: `/v3/users/nova` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:13.967465 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e81b2d-7a96-41c2-8408-6ad889c3004e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:13.968956 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e81b2d-7a96-41c2-8408-6ad889c3004e None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:13.981061 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-94e81b2d-7a96-41c2-8408-6ad889c3004e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 00:36:13.981061 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:13.981061 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:13.981061 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:13.981061 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:13.981061 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:13.981061 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:13.981061 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:13.981061 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:13.981061 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:13.981061 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:13.981061 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:13.981061 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:36:13.981061 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:36:13.981061 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:36:13.981061 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:36:13.982111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:13.982111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:13.982111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:36:13.982111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:36:13.982111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:36:13.982111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:36:13.982111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:13.982111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:13.982111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:13.982111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:13.982111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:13.982111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:13.982111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:13.982111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:13.982111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:13.982111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:13.982111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:13.982111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:13.983110 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:13.983110 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:13.983110 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:36:13.983110 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:36:13.983110 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:36:13.983110 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:36:13.983110 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:36:13.983110 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:36:13.983110 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 00:36:13.983110 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:36:13.983110 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 529/1068] 10.222.0.59 () {56 vars in 1093 bytes} [Fri Aug 26 00:36:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:36:13.989320 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36e228-41eb-4170-bdc8-c3869fac4a48 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:14.057055 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e81b2d-7a96-41c2-8408-6ad889c3004e None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:14.098143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36e228-41eb-4170-bdc8-c3869fac4a48 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:14.128835 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:14.134775 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bd1c0-ac87-489d-959c-c0f308341b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:14.135376 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bd1c0-ac87-489d-959c-c0f308341b1e None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:14.135708 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bd1c0-ac87-489d-959c-c0f308341b1e None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:14.135996 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bd1c0-ac87-489d-959c-c0f308341b1e None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:14.136761 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9bd1c0-ac87-489d-959c-c0f308341b1e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:14.137076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9bd1c0-ac87-489d-959c-c0f308341b1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:14.139352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9bd1c0-ac87-489d-959c-c0f308341b1e None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:14.157131 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 540/1069] 10.222.0.59 () {56 vars in 1133 bytes} [Fri Aug 26 00:36:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:14.162647 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bd1c0-ac87-489d-959c-c0f308341b1e None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:14.165648 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e81b2d-7a96-41c2-8408-6ad889c3004e None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:14.202345 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3da796-9129-4a8c-b065-e8b032127b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:14.202979 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3da796-9129-4a8c-b065-e8b032127b62 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:14.203261 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3da796-9129-4a8c-b065-e8b032127b62 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:14.203535 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3da796-9129-4a8c-b065-e8b032127b62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:14.211376 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:14.215187 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3da796-9129-4a8c-b065-e8b032127b62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:36:14.220564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-0b3da796-9129-4a8c-b065-e8b032127b62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:36:14.220564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:14.220564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:14.220564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:14.220564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:14.220564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:14.220564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:14.220564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:14.220564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:14.220564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:14.220564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:14.220564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:14.220564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:36:14.220564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:36:14.220564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:36:14.220564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:36:14.227689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:36:14.227689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:36:14.227689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:36:14.227689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:36:14.227689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:14.227689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:14.227689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:14.227689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:14.227689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:14.227689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:14.227689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:14.227689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:14.227689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:14.227689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:14.227689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:14.227689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:14.228696 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:14.228696 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:14.228696 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:14.228696 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:14.228696 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:36:14.228696 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:36:14.228696 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:36:14.228696 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:36:14.228696 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:36:14.228696 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:36:14.228696 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:36:14.228696 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:36:14.228696 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 530/1070] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:36:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:36:14.231529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3da796-9129-4a8c-b065-e8b032127b62 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:14.287417 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bd1c0-ac87-489d-959c-c0f308341b1e None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:14.329741 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369cd8c5-f1f8-413f-9b67-d33b53f43082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:14.330382 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369cd8c5-f1f8-413f-9b67-d33b53f43082 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:14.330664 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369cd8c5-f1f8-413f-9b67-d33b53f43082 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:14.330936 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369cd8c5-f1f8-413f-9b67-d33b53f43082 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:14.342631 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369cd8c5-f1f8-413f-9b67-d33b53f43082 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:14.344082 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369cd8c5-f1f8-413f-9b67-d33b53f43082 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:14.344192 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3da796-9129-4a8c-b065-e8b032127b62 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:14.358627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-369cd8c5-f1f8-413f-9b67-d33b53f43082 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:36:14.358627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:14.358627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:14.358627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:14.358627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:14.358627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:14.358627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:14.358627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:14.358627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:14.358627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:14.358627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:14.358627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:14.358627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:36:14.358627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:36:14.358627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:36:14.358627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:36:14.359817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:14.359817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:14.359817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:14.359817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:14.359817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:14.359817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:14.359817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:14.359817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:14.359817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:14.359817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:14.359817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:14.359817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:14.359817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:14.359817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:14.359817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:14.359817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:14.359817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:36:14.360779 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:36:14.360779 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:36:14.360779 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:36:14.360779 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 541/1071] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:36:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:36:14.365589 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369cd8c5-f1f8-413f-9b67-d33b53f43082 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:14.386096 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87682684-8849-430f-b3ba-151334e7001e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:14.386695 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87682684-8849-430f-b3ba-151334e7001e None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:14.386973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87682684-8849-430f-b3ba-151334e7001e None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:14.387244 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87682684-8849-430f-b3ba-151334e7001e None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:14.388063 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87682684-8849-430f-b3ba-151334e7001e None admin] RBAC: Adding query filter params (name=admin) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:14.388368 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87682684-8849-430f-b3ba-151334e7001e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:14.390352 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87682684-8849-430f-b3ba-151334e7001e None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:14.399290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 531/1072] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:36:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:14.405888 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87682684-8849-430f-b3ba-151334e7001e None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:14.417013 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:14.476076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:14.480686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369cd8c5-f1f8-413f-9b67-d33b53f43082 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:14.517227 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df37e57-c540-46c1-ac1f-37be66b1313d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:14.517839 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df37e57-c540-46c1-ac1f-37be66b1313d None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:14.517949 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87682684-8849-430f-b3ba-151334e7001e None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:14.518132 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df37e57-c540-46c1-ac1f-37be66b1313d None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:14.518382 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df37e57-c540-46c1-ac1f-37be66b1313d None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:14.519017 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df37e57-c540-46c1-ac1f-37be66b1313d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:14.519373 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df37e57-c540-46c1-ac1f-37be66b1313d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:14.520765 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df37e57-c540-46c1-ac1f-37be66b1313d None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:14.538227 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 542/1073] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:14.539293 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df37e57-c540-46c1-ac1f-37be66b1313d None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:14.557672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54550041-434f-4657-84fd-4a892fd82644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:14.558380 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54550041-434f-4657-84fd-4a892fd82644 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:14.558663 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54550041-434f-4657-84fd-4a892fd82644 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:14.558945 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54550041-434f-4657-84fd-4a892fd82644 None admin] PATH_INFO: `/v3/users/3927365b4b08436ba8b5e4f71690482d` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:14.560048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54550041-434f-4657-84fd-4a892fd82644 None admin] RBAC: Authorizing `identity:get_user(user_id=3927365b4b08436ba8b5e4f71690482d)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:14.561570 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54550041-434f-4657-84fd-4a892fd82644 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:14.567985 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 532/1074] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:36:14 2022] GET /identity/v3/users/3927365b4b08436ba8b5e4f71690482d => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:36:14.568811 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54550041-434f-4657-84fd-4a892fd82644 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:14.641889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:14.645248 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df37e57-c540-46c1-ac1f-37be66b1313d None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:14.675768 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54550041-434f-4657-84fd-4a892fd82644 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:14.679142 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d148d95-fdb6-4f03-a2b5-b40753c08cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:14.679776 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d148d95-fdb6-4f03-a2b5-b40753c08cdd None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:14.680060 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d148d95-fdb6-4f03-a2b5-b40753c08cdd None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:14.680335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d148d95-fdb6-4f03-a2b5-b40753c08cdd None admin] PATH_INFO: `/v3/projects/service` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:14.687959 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d148d95-fdb6-4f03-a2b5-b40753c08cdd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:14.689435 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d148d95-fdb6-4f03-a2b5-b40753c08cdd None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:14.701802 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-3d148d95-fdb6-4f03-a2b5-b40753c08cdd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:36:14.701802 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:14.701802 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:14.701802 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:14.701802 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:14.701802 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:14.701802 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:14.701802 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:14.701802 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:14.701802 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:14.701802 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:14.701802 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:14.701802 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:36:14.701802 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:36:14.701802 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:36:14.702753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:36:14.702753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:14.702753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:14.702753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:14.702753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:14.702753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:14.702753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:14.702753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:14.702753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:14.702753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:14.702753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:14.702753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:14.702753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:14.702753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:14.702753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:14.702753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:14.702753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:14.703686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:36:14.703686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:36:14.703686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:36:14.703686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:36:14.703686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:36:14.703686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:36:14.703686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:36:14.703686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:36:14.703686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 543/1075] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:36:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:36:14.707536 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:14.709003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d148d95-fdb6-4f03-a2b5-b40753c08cdd None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:14.713459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c7bfc2-b3df-409e-93bb-e2d5d9a70eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:14.714093 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c7bfc2-b3df-409e-93bb-e2d5d9a70eb0 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:14.714368 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c7bfc2-b3df-409e-93bb-e2d5d9a70eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:14.714633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c7bfc2-b3df-409e-93bb-e2d5d9a70eb0 None admin] PATH_INFO: `/v3/projects/abf5cf909af24d2281230165a0bd8526` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:14.715768 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c7bfc2-b3df-409e-93bb-e2d5d9a70eb0 None admin] RBAC: Authorizing `identity:get_project(project_id=abf5cf909af24d2281230165a0bd8526)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:14.717786 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c7bfc2-b3df-409e-93bb-e2d5d9a70eb0 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:14.719590 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 533/1076] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:36:14 2022] GET /identity/v3/projects/abf5cf909af24d2281230165a0bd8526 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:14.725798 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c7bfc2-b3df-409e-93bb-e2d5d9a70eb0 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:14.816105 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d148d95-fdb6-4f03-a2b5-b40753c08cdd None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:14.830946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c7bfc2-b3df-409e-93bb-e2d5d9a70eb0 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:14.851137 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33848dd-fd07-429f-8cb3-6c67031cd83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:14.851752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33848dd-fd07-429f-8cb3-6c67031cd83c None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:14.852036 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33848dd-fd07-429f-8cb3-6c67031cd83c None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:14.852307 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33848dd-fd07-429f-8cb3-6c67031cd83c None admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:14.853033 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33848dd-fd07-429f-8cb3-6c67031cd83c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:14.853340 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33848dd-fd07-429f-8cb3-6c67031cd83c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:14.854768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33848dd-fd07-429f-8cb3-6c67031cd83c None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:14.864360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995243d5-8c1b-4499-bcba-528a2ec2cda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:14.865037 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995243d5-8c1b-4499-bcba-528a2ec2cda0 None admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:14.865313 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995243d5-8c1b-4499-bcba-528a2ec2cda0 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:14.865583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995243d5-8c1b-4499-bcba-528a2ec2cda0 None admin] PATH_INFO: `/v3/projects/abf5cf909af24d2281230165a0bd8526/users/3927365b4b08436ba8b5e4f71690482d/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:14.866681 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 544/1077] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:36:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:14.871718 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdfd26-2e6a-4139-ad03-68f8fa0dc267 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:14.871718 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdfd26-2e6a-4139-ad03-68f8fa0dc267 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:14.871718 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdfd26-2e6a-4139-ad03-68f8fa0dc267 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:14.871718 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 545/1078] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:36:14.872328 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995243d5-8c1b-4499-bcba-528a2ec2cda0 None admin] RBAC: Authorizing `identity:create_grant(project_id=abf5cf909af24d2281230165a0bd8526, user_id=3927365b4b08436ba8b5e4f71690482d, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:14.872556 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fdfd26-2e6a-4139-ad03-68f8fa0dc267 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:14.874536 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995243d5-8c1b-4499-bcba-528a2ec2cda0 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:14.894904 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:14.943908 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 534/1079] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Aug 26 00:36:14 2022] PUT /identity/v3/projects/abf5cf909af24d2281230165a0bd8526/users/3927365b4b08436ba8b5e4f71690482d/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:36:14.945174 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d8084-e97b-4179-afd1-5df9d046206c None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:14.945462 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d8084-e97b-4179-afd1-5df9d046206c None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:14.945736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d8084-e97b-4179-afd1-5df9d046206c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:14.965424 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-704d8084-e97b-4179-afd1-5df9d046206c None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:14.966635 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-704d8084-e97b-4179-afd1-5df9d046206c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:14.989872 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fdfd26-2e6a-4139-ad03-68f8fa0dc267 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:15.031154 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:15.034891 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdbc0a-405f-4e85-9769-505434e24c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:15.035585 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdbc0a-405f-4e85-9769-505434e24c6c None admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:15.035858 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdbc0a-405f-4e85-9769-505434e24c6c None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:15.036141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdbc0a-405f-4e85-9769-505434e24c6c None admin] PATH_INFO: `/v3/projects/fde7ac3ed472433ab3bba56e5e091510/users/0602593263ca4b05a657d5b7697f932f/roles/91940e122e2f46e3818c5f11f4fca9e9` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:15.038469 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfdbc0a-405f-4e85-9769-505434e24c6c None admin] RBAC: Authorizing `identity:create_grant(project_id=fde7ac3ed472433ab3bba56e5e091510, user_id=0602593263ca4b05a657d5b7697f932f, role_id=91940e122e2f46e3818c5f11f4fca9e9)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:15.039975 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfdbc0a-405f-4e85-9769-505434e24c6c None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:15.047391 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 535/1080] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:36:15.124159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 546/1081] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Aug 26 00:36:14 2022] PUT /identity/v3/projects/fde7ac3ed472433ab3bba56e5e091510/users/0602593263ca4b05a657d5b7697f932f/roles/91940e122e2f46e3818c5f11f4fca9e9 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:36:15.318352 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d8084-e97b-4179-afd1-5df9d046206c None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:15.455443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d8084-e97b-4179-afd1-5df9d046206c None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:15.492084 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae34ea7-e2aa-4fd9-af3a-71c91cbb0831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:15.492815 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae34ea7-e2aa-4fd9-af3a-71c91cbb0831 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:15.493167 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae34ea7-e2aa-4fd9-af3a-71c91cbb0831 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:15.493505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae34ea7-e2aa-4fd9-af3a-71c91cbb0831 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:15.548911 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:15.561130 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae34ea7-e2aa-4fd9-af3a-71c91cbb0831 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:15.562777 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae34ea7-e2aa-4fd9-af3a-71c91cbb0831 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:15.665211 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 536/1082] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:36:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 00:36:15.671498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdbc0a-405f-4e85-9769-505434e24c6c None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:15.750712 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:15.769827 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdbc0a-405f-4e85-9769-505434e24c6c None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:15.806922 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec8eff-479e-4021-8417-c442ce04dae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:15.807640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec8eff-479e-4021-8417-c442ce04dae6 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:15.808004 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec8eff-479e-4021-8417-c442ce04dae6 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:15.808343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec8eff-479e-4021-8417-c442ce04dae6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:15.818226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec8eff-479e-4021-8417-c442ce04dae6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:15.819766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec8eff-479e-4021-8417-c442ce04dae6 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:15.835009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-90ec8eff-479e-4021-8417-c442ce04dae6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:36:15.835009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:15.835009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:15.835009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:15.835009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:15.835009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:15.835009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:15.835009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:15.835009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:15.835009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:15.835009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:15.835009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:15.835009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:36:15.835009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:36:15.835009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:36:15.835009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:36:15.835995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:15.835995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:15.835995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:15.835995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:15.835995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:15.835995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:15.835995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:15.835995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:15.835995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:15.835995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:15.835995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:15.835995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:15.835995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:15.835995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:15.835995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:15.835995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:15.835995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:36:15.837317 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:36:15.837317 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:36:15.837317 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:36:15.840781 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 547/1083] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:36:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:36:15.842579 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae34ea7-e2aa-4fd9-af3a-71c91cbb0831 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:15.928049 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:15.939840 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae34ea7-e2aa-4fd9-af3a-71c91cbb0831 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:15.974294 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87510c57-5e50-489f-8df3-ca4ab83f7aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:15.974905 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87510c57-5e50-489f-8df3-ca4ab83f7aa2 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:15.975186 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87510c57-5e50-489f-8df3-ca4ab83f7aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:15.975459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87510c57-5e50-489f-8df3-ca4ab83f7aa2 None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:15.976102 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87510c57-5e50-489f-8df3-ca4ab83f7aa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:15.976404 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87510c57-5e50-489f-8df3-ca4ab83f7aa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:15.977792 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87510c57-5e50-489f-8df3-ca4ab83f7aa2 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:15.990066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 537/1084] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:16.886711 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f4f15-f3c5-4e32-abbe-2c9c7a420cd3 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:16.887245 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f4f15-f3c5-4e32-abbe-2c9c7a420cd3 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:16.887245 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f4f15-f3c5-4e32-abbe-2c9c7a420cd3 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:16.889182 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 548/1085] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:36:16.894485 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01222cc-dd79-4104-974b-f92411409bb4 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:16.894931 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01222cc-dd79-4104-974b-f92411409bb4 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:16.895023 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01222cc-dd79-4104-974b-f92411409bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:16.912757 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-c01222cc-dd79-4104-974b-f92411409bb4 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:16.914036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-c01222cc-dd79-4104-974b-f92411409bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:16.977442 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 538/1086] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:36:17.048785 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f9da0-8dd0-4755-ad46-a2d34dc3422c None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:17.049075 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f9da0-8dd0-4755-ad46-a2d34dc3422c None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:17.049355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f9da0-8dd0-4755-ad46-a2d34dc3422c None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:17.050691 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 549/1087] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:36:17.055846 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9f11a-9de3-496b-b2f3-497cc007de18 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:17.056133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9f11a-9de3-496b-b2f3-497cc007de18 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:17.056409 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9f11a-9de3-496b-b2f3-497cc007de18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:17.074945 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-32b9f11a-9de3-496b-b2f3-497cc007de18 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:17.076831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-32b9f11a-9de3-496b-b2f3-497cc007de18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:17.142319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 539/1088] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:36:17.240537 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f9da0-8dd0-4755-ad46-a2d34dc3422c None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:17.339535 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f9da0-8dd0-4755-ad46-a2d34dc3422c None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:17.369178 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:17.374461 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742db1c-52c8-4be7-ba61-dc8aa4038dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:17.375080 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742db1c-52c8-4be7-ba61-dc8aa4038dbb None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:17.375360 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742db1c-52c8-4be7-ba61-dc8aa4038dbb None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:17.375631 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742db1c-52c8-4be7-ba61-dc8aa4038dbb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:17.385389 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8742db1c-52c8-4be7-ba61-dc8aa4038dbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:36:17.390158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-8742db1c-52c8-4be7-ba61-dc8aa4038dbb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:36:17.390158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:17.390158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:17.390158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:17.390158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:17.390158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:17.390158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:17.390158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:17.390158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:17.390158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:17.390158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:17.390158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:17.390158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:36:17.390158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:36:17.390158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:36:17.390158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:36:17.396653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:36:17.396653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:36:17.396653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:36:17.396653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:36:17.396653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:17.396653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:17.396653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:17.396653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:17.396653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:17.396653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:17.396653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:17.396653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:17.396653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:17.396653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:17.396653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:17.396653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:17.397583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:17.397583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:17.397583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:17.397583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:17.397583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:36:17.397583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:36:17.397583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:36:17.397583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:36:17.397583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:36:17.397583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:36:17.397583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:36:17.397583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:36:17.397583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 550/1089] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:36:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:36:17.398389 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9f11a-9de3-496b-b2f3-497cc007de18 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:17.408354 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:17.408539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742db1c-52c8-4be7-ba61-dc8aa4038dbb None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:17.500101 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9f11a-9de3-496b-b2f3-497cc007de18 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:17.517653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742db1c-52c8-4be7-ba61-dc8aa4038dbb None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:17.534030 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb4a96-1f21-4816-bcda-667a6f56505e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:17.534633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb4a96-1f21-4816-bcda-667a6f56505e None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:17.534910 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb4a96-1f21-4816-bcda-667a6f56505e None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:17.535178 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb4a96-1f21-4816-bcda-667a6f56505e None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:17.535929 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb4a96-1f21-4816-bcda-667a6f56505e None admin] RBAC: Adding query filter params (name=admin) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:17.536136 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb4a96-1f21-4816-bcda-667a6f56505e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:17.537539 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb4a96-1f21-4816-bcda-667a6f56505e None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:17.546100 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 540/1090] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:36:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:17.548609 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab63f7-1d30-4701-9a1d-2d0669656967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:17.549251 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab63f7-1d30-4701-9a1d-2d0669656967 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:17.549533 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab63f7-1d30-4701-9a1d-2d0669656967 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:17.549808 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab63f7-1d30-4701-9a1d-2d0669656967 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:17.551844 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb4a96-1f21-4816-bcda-667a6f56505e None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:17.560129 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ab63f7-1d30-4701-9a1d-2d0669656967 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:36:17.568884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-e9ab63f7-1d30-4701-9a1d-2d0669656967 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 00:36:17.568884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:17.568884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:17.568884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:17.568884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:17.568884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:17.568884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:17.568884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:17.568884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:17.568884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:17.568884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:17.568884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:17.568884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:36:17.568884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:36:17.568884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:36:17.568884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:36:17.570123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:36:17.570123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:36:17.570123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:36:17.570123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:36:17.570123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:17.570123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:17.570123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:17.570123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:17.570123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:17.570123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:17.570123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:17.570123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:17.570123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:17.570123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:17.570123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:17.570123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:17.571008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:17.571008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:17.571008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:17.571008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:17.571008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:36:17.571008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:36:17.571008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:36:17.571008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:36:17.571008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:36:17.571008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:36:17.571008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 00:36:17.571008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:36:17.571008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 551/1091] 10.222.0.59 () {56 vars in 1120 bytes} [Fri Aug 26 00:36:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:36:17.576513 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab63f7-1d30-4701-9a1d-2d0669656967 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:17.630986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:17.635625 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:17.666843 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb4a96-1f21-4816-bcda-667a6f56505e None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:17.679196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab63f7-1d30-4701-9a1d-2d0669656967 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:17.701424 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f72aa-9749-481a-a5c4-ec5ac9b5ae7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:17.702058 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f72aa-9749-481a-a5c4-ec5ac9b5ae7d None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:17.702337 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f72aa-9749-481a-a5c4-ec5ac9b5ae7d None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:17.702609 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f72aa-9749-481a-a5c4-ec5ac9b5ae7d None admin] PATH_INFO: `/v3/users/3927365b4b08436ba8b5e4f71690482d` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:17.703736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f72aa-9749-481a-a5c4-ec5ac9b5ae7d None admin] RBAC: Authorizing `identity:get_user(user_id=3927365b4b08436ba8b5e4f71690482d)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:17.705195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f72aa-9749-481a-a5c4-ec5ac9b5ae7d None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:17.706772 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 541/1092] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:36:17 2022] GET /identity/v3/users/3927365b4b08436ba8b5e4f71690482d => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:36:17.712929 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f72aa-9749-481a-a5c4-ec5ac9b5ae7d None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:17.713744 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1a015-8291-482f-9363-06b68ca48318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:17.714364 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1a015-8291-482f-9363-06b68ca48318 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:17.714646 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1a015-8291-482f-9363-06b68ca48318 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:17.714920 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1a015-8291-482f-9363-06b68ca48318 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:17.715592 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1a015-8291-482f-9363-06b68ca48318 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:17.715894 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1a015-8291-482f-9363-06b68ca48318 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:17.717849 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1a015-8291-482f-9363-06b68ca48318 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:17.726101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 552/1093] 10.222.0.59 () {56 vars in 1115 bytes} [Fri Aug 26 00:36:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:36:17.731698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1a015-8291-482f-9363-06b68ca48318 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:17.812269 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f72aa-9749-481a-a5c4-ec5ac9b5ae7d None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:17.837007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1a015-8291-482f-9363-06b68ca48318 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:17.846942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f1718-285e-4122-873c-57d005db6b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:17.847550 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f1718-285e-4122-873c-57d005db6b62 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:17.847839 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f1718-285e-4122-873c-57d005db6b62 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:17.848106 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f1718-285e-4122-873c-57d005db6b62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:17.853002 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:17.856793 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:17.874047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca207533-f330-474f-9a67-f143491a5e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:17.874784 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca207533-f330-474f-9a67-f143491a5e00 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:17.875125 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca207533-f330-474f-9a67-f143491a5e00 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:17.876855 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca207533-f330-474f-9a67-f143491a5e00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:17.887023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca207533-f330-474f-9a67-f143491a5e00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:17.888616 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca207533-f330-474f-9a67-f143491a5e00 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:17.908287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-ca207533-f330-474f-9a67-f143491a5e00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:36:17.908287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:17.908287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:17.908287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:17.908287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:17.908287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:17.908287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:17.908287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:17.908287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:17.908287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:17.908287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:17.908287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:17.908287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:36:17.908287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:36:17.908287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:36:17.908287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:36:17.913663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:17.913663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:17.913663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:17.913663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:17.913663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:17.913663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:17.913663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:17.913663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:17.913663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:17.913663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:17.913663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:17.913663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:17.913663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:17.913663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:17.913663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:17.913663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:17.913663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:36:17.914909 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:36:17.914909 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:36:17.914909 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:36:17.914909 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 553/1094] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:36:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:36:17.915772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca207533-f330-474f-9a67-f143491a5e00 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:17.919699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f1718-285e-4122-873c-57d005db6b62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:17.921844 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f1718-285e-4122-873c-57d005db6b62 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:18.019477 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca207533-f330-474f-9a67-f143491a5e00 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:18.029991 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 542/1095] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:36:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 00:36:18.035851 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f1718-285e-4122-873c-57d005db6b62 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:18.054158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935d1c3-61cd-4244-9885-89ba0255fe6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:18.054774 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935d1c3-61cd-4244-9885-89ba0255fe6d None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:18.055063 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935d1c3-61cd-4244-9885-89ba0255fe6d None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:18.055339 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935d1c3-61cd-4244-9885-89ba0255fe6d None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:18.055984 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935d1c3-61cd-4244-9885-89ba0255fe6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:18.056292 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935d1c3-61cd-4244-9885-89ba0255fe6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:18.057694 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935d1c3-61cd-4244-9885-89ba0255fe6d None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:18.071126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 554/1096] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:36:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:18.074181 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:18.077345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935d1c3-61cd-4244-9885-89ba0255fe6d None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:18.106261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:18.135105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f1718-285e-4122-873c-57d005db6b62 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:18.170375 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642d58d-71a0-4200-bebc-4b8dda586707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:18.170927 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642d58d-71a0-4200-bebc-4b8dda586707 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:18.171209 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642d58d-71a0-4200-bebc-4b8dda586707 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:18.171484 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642d58d-71a0-4200-bebc-4b8dda586707 None admin] PATH_INFO: `/v3/projects/abf5cf909af24d2281230165a0bd8526` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:18.172773 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5642d58d-71a0-4200-bebc-4b8dda586707 None admin] RBAC: Authorizing `identity:get_project(project_id=abf5cf909af24d2281230165a0bd8526)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:18.182264 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5642d58d-71a0-4200-bebc-4b8dda586707 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:18.183917 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 543/1097] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:36:18 2022] GET /identity/v3/projects/abf5cf909af24d2281230165a0bd8526 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:18.184014 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935d1c3-61cd-4244-9885-89ba0255fe6d None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:18.191891 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642d58d-71a0-4200-bebc-4b8dda586707 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:18.220861 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379816ae-164c-4d0c-b805-602d031b3cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:18.221500 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379816ae-164c-4d0c-b805-602d031b3cb9 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:18.221779 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379816ae-164c-4d0c-b805-602d031b3cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:18.222056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379816ae-164c-4d0c-b805-602d031b3cb9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:18.236099 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379816ae-164c-4d0c-b805-602d031b3cb9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:18.237582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379816ae-164c-4d0c-b805-602d031b3cb9 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:18.249385 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-379816ae-164c-4d0c-b805-602d031b3cb9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 00:36:18.249385 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:18.249385 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:18.249385 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:18.249385 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:18.249385 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:18.249385 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:18.249385 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:18.249385 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:18.249385 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:18.249385 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:18.249385 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:18.249385 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:36:18.249385 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:36:18.249385 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:36:18.249385 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:36:18.250581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:18.250581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:18.250581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:36:18.250581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:36:18.250581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:36:18.250581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:36:18.250581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:18.250581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:18.250581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:18.250581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:18.250581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:18.250581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:18.250581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:18.250581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:18.250581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:18.250581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:18.250581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:18.250581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:18.251724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:18.251724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:18.251724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:36:18.251724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:36:18.251724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:36:18.251724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:36:18.251724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:36:18.251724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:36:18.251724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 00:36:18.251724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:36:18.251724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 555/1098] 10.222.0.59 () {56 vars in 1093 bytes} [Fri Aug 26 00:36:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:36:18.256243 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379816ae-164c-4d0c-b805-602d031b3cb9 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:18.303319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642d58d-71a0-4200-bebc-4b8dda586707 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:18.311218 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:18.343347 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71956b-54d8-44c3-879d-b3445aa46ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:18.343955 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71956b-54d8-44c3-879d-b3445aa46ed9 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:18.344238 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71956b-54d8-44c3-879d-b3445aa46ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:18.344537 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71956b-54d8-44c3-879d-b3445aa46ed9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:18.345264 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71956b-54d8-44c3-879d-b3445aa46ed9 None admin] RBAC: Adding query filter params (role.id=1a33e3c8cb8347feb9498595886cc621, scope.project.id=abf5cf909af24d2281230165a0bd8526, user.id=3927365b4b08436ba8b5e4f71690482d) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:18.345576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71956b-54d8-44c3-879d-b3445aa46ed9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:18.347776 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71956b-54d8-44c3-879d-b3445aa46ed9 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:18.350445 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:18.355416 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 544/1099] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:36:18 2022] GET /identity/v3/role_assignments?user.id=3927365b4b08436ba8b5e4f71690482d&scope.project.id=abf5cf909af24d2281230165a0bd8526&role.id=1a33e3c8cb8347feb9498595886cc621 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:36:18.365351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379816ae-164c-4d0c-b805-602d031b3cb9 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:18.406678 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b936c4-ef15-4ce7-9b5e-2dee9372529d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:18.407324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b936c4-ef15-4ce7-9b5e-2dee9372529d None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:18.407605 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b936c4-ef15-4ce7-9b5e-2dee9372529d None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:18.407879 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b936c4-ef15-4ce7-9b5e-2dee9372529d None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:18.408669 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b936c4-ef15-4ce7-9b5e-2dee9372529d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:18.408977 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b936c4-ef15-4ce7-9b5e-2dee9372529d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:18.410458 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b936c4-ef15-4ce7-9b5e-2dee9372529d None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:18.429930 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 556/1100] 10.222.0.59 () {56 vars in 1133 bytes} [Fri Aug 26 00:36:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:36:18.436750 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71956b-54d8-44c3-879d-b3445aa46ed9 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:18.551629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71956b-54d8-44c3-879d-b3445aa46ed9 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:18.602108 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec07005-a629-421a-9c2c-78352c0550cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:18.603119 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec07005-a629-421a-9c2c-78352c0550cd None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:18.603610 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec07005-a629-421a-9c2c-78352c0550cd None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:18.604081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec07005-a629-421a-9c2c-78352c0550cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:18.651754 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:18.685703 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec07005-a629-421a-9c2c-78352c0550cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:18.687093 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec07005-a629-421a-9c2c-78352c0550cd None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:18.785081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 545/1101] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:36:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 00:36:18.791078 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b936c4-ef15-4ce7-9b5e-2dee9372529d None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:18.887671 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b936c4-ef15-4ce7-9b5e-2dee9372529d None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:18.920669 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f7c9c-2697-4a94-a665-c9ccde796c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:18.921295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f7c9c-2697-4a94-a665-c9ccde796c84 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:18.921568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f7c9c-2697-4a94-a665-c9ccde796c84 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:18.921837 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f7c9c-2697-4a94-a665-c9ccde796c84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:18.931459 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f7c9c-2697-4a94-a665-c9ccde796c84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:18.932919 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f7c9c-2697-4a94-a665-c9ccde796c84 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:18.935895 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:18.947713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-158f7c9c-2697-4a94-a665-c9ccde796c84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:36:18.947713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:18.947713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:18.947713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:18.947713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:18.947713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:18.947713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:18.947713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:18.947713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:18.947713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:18.947713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:18.947713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:18.947713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:36:18.947713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:36:18.947713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:36:18.947713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:36:18.955179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:18.955179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:18.955179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:18.955179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:18.955179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:18.955179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:18.955179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:18.955179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:18.955179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:18.955179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:18.955179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:18.955179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:18.955179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:18.955179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:18.955179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:18.955179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:18.955179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:36:18.956114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:36:18.956114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:36:18.956114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:36:18.956114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 557/1102] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:36:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:36:18.956345 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec07005-a629-421a-9c2c-78352c0550cd None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:19.022013 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:19.053106 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec07005-a629-421a-9c2c-78352c0550cd None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:19.109690 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7577e1-9b0b-466b-ab86-9c9e282636a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:19.110286 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7577e1-9b0b-466b-ab86-9c9e282636a1 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:19.110573 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7577e1-9b0b-466b-ab86-9c9e282636a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:19.110841 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7577e1-9b0b-466b-ab86-9c9e282636a1 None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:19.111476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7577e1-9b0b-466b-ab86-9c9e282636a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:19.111775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7577e1-9b0b-466b-ab86-9c9e282636a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:19.113163 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7577e1-9b0b-466b-ab86-9c9e282636a1 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:19.124962 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 546/1103] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:19.130621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f7c9c-2697-4a94-a665-c9ccde796c84 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:19.227735 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f7c9c-2697-4a94-a665-c9ccde796c84 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:19.259987 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdbed07-fa4d-407e-bf51-ff98078a974a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:19.260646 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbed07-fa4d-407e-bf51-ff98078a974a None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:19.260930 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbed07-fa4d-407e-bf51-ff98078a974a None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:19.261200 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbed07-fa4d-407e-bf51-ff98078a974a None admin] PATH_INFO: `/v3/projects/service` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:19.268221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdbed07-fa4d-407e-bf51-ff98078a974a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:19.269678 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdbed07-fa4d-407e-bf51-ff98078a974a None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:19.281126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-5fdbed07-fa4d-407e-bf51-ff98078a974a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:36:19.281126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:19.281126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:19.281126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:19.281126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:19.281126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:19.281126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:19.281126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:19.281126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:19.281126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:19.281126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:19.281126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:19.281126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:36:19.281126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:36:19.281126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:36:19.282184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:36:19.282184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:19.282184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:19.282184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:19.282184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:19.282184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:19.282184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:19.282184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:19.282184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:19.282184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:19.282184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:19.282184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:19.282184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:19.282184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:19.282184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:19.282184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:19.282184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:19.283094 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:36:19.283094 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:36:19.283094 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:36:19.283094 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:36:19.283094 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:36:19.283094 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:36:19.283094 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:36:19.283094 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:36:19.283094 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 558/1104] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:36:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:36:19.288193 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7577e1-9b0b-466b-ab86-9c9e282636a1 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:19.384259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7577e1-9b0b-466b-ab86-9c9e282636a1 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:19.416532 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:19.422403 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e68f8-d912-45ab-a27e-fe8ed9858497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:19.423012 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e68f8-d912-45ab-a27e-fe8ed9858497 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:19.423292 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e68f8-d912-45ab-a27e-fe8ed9858497 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:19.423561 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e68f8-d912-45ab-a27e-fe8ed9858497 None admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:19.424311 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e68f8-d912-45ab-a27e-fe8ed9858497 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:19.424665 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e68f8-d912-45ab-a27e-fe8ed9858497 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:19.426143 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e68f8-d912-45ab-a27e-fe8ed9858497 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:19.438172 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 547/1105] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:36:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:19.444397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdbed07-fa4d-407e-bf51-ff98078a974a None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:19.502477 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:19.545364 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdbed07-fa4d-407e-bf51-ff98078a974a None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:19.578233 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94364f95-cfa6-4f8b-b07a-7f4a138da77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:19.578836 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94364f95-cfa6-4f8b-b07a-7f4a138da77c None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:19.579118 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94364f95-cfa6-4f8b-b07a-7f4a138da77c None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:19.579393 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94364f95-cfa6-4f8b-b07a-7f4a138da77c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:19.580115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94364f95-cfa6-4f8b-b07a-7f4a138da77c None admin] RBAC: Adding query filter params (role.id=91940e122e2f46e3818c5f11f4fca9e9, scope.project.id=fde7ac3ed472433ab3bba56e5e091510, user.id=0602593263ca4b05a657d5b7697f932f) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:19.580424 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94364f95-cfa6-4f8b-b07a-7f4a138da77c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:19.581876 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94364f95-cfa6-4f8b-b07a-7f4a138da77c None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:19.587480 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 559/1106] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:36:19 2022] GET /identity/v3/role_assignments?user.id=0602593263ca4b05a657d5b7697f932f&scope.project.id=fde7ac3ed472433ab3bba56e5e091510&role.id=91940e122e2f46e3818c5f11f4fca9e9 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:20.315193 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e7c81-17cd-47f2-9d8f-9ca37e5f8b60 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:20.315732 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e7c81-17cd-47f2-9d8f-9ca37e5f8b60 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:20.315732 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e7c81-17cd-47f2-9d8f-9ca37e5f8b60 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:20.317109 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 548/1107] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:36:20.322794 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa4565-41bc-4124-820a-5f6399b0cb83 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:20.323244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa4565-41bc-4124-820a-5f6399b0cb83 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:20.323466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa4565-41bc-4124-820a-5f6399b0cb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:20.343442 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-49aa4565-41bc-4124-820a-5f6399b0cb83 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:20.344682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-49aa4565-41bc-4124-820a-5f6399b0cb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:20.413901 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 560/1108] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:36:20.688832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e7c81-17cd-47f2-9d8f-9ca37e5f8b60 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:20.789741 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e7c81-17cd-47f2-9d8f-9ca37e5f8b60 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:20.821769 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75dca57-c693-4076-adb7-45a5005069b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:20.822384 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75dca57-c693-4076-adb7-45a5005069b8 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:20.822660 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75dca57-c693-4076-adb7-45a5005069b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:20.822924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75dca57-c693-4076-adb7-45a5005069b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:20.824007 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75dca57-c693-4076-adb7-45a5005069b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:20.825444 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75dca57-c693-4076-adb7-45a5005069b8 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:20.827542 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 549/1109] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:36:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:20.833227 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa4565-41bc-4124-820a-5f6399b0cb83 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:20.879700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:20.927117 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa4565-41bc-4124-820a-5f6399b0cb83 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:20.960037 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a88832-1ee9-4969-ae36-3d5ed7a28008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:20.960614 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a88832-1ee9-4969-ae36-3d5ed7a28008 None admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:20.960894 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a88832-1ee9-4969-ae36-3d5ed7a28008 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:20.961168 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a88832-1ee9-4969-ae36-3d5ed7a28008 None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:20.961819 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a88832-1ee9-4969-ae36-3d5ed7a28008 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:20.963239 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a88832-1ee9-4969-ae36-3d5ed7a28008 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:21.059058 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 561/1110] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Aug 26 00:36:20 2022] POST /identity/v3/users => generated 286 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:36:22.971571 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3617b43-54d6-4865-a831-60c9500148a4 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:22.972211 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3617b43-54d6-4865-a831-60c9500148a4 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:22.972211 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3617b43-54d6-4865-a831-60c9500148a4 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:22.973591 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 550/1111] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:36:22.978666 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651c775-ca80-4061-ae62-a0d5fc2a988c None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:22.979088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651c775-ca80-4061-ae62-a0d5fc2a988c None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:22.979194 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651c775-ca80-4061-ae62-a0d5fc2a988c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:22.996630 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-f651c775-ca80-4061-ae62-a0d5fc2a988c None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:22.998330 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f651c775-ca80-4061-ae62-a0d5fc2a988c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:23.062213 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 562/1112] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:36:23.331257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3617b43-54d6-4865-a831-60c9500148a4 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:23.388782 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:23.429064 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3617b43-54d6-4865-a831-60c9500148a4 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:23.462480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a120b-80fe-4d36-8527-3ea2991cf24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:23.463106 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a120b-80fe-4d36-8527-3ea2991cf24b None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:23.463390 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a120b-80fe-4d36-8527-3ea2991cf24b None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:23.463664 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a120b-80fe-4d36-8527-3ea2991cf24b None admin] PATH_INFO: `/v3/roles/423aa6ac8b05402da7587e83e5c4ce09` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:23.465078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a120b-80fe-4d36-8527-3ea2991cf24b None admin] RBAC: Authorizing `identity:get_role(role_id=423aa6ac8b05402da7587e83e5c4ce09)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:23.466472 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a120b-80fe-4d36-8527-3ea2991cf24b None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:23.467789 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 551/1113] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:36:23 2022] GET /identity/v3/roles/423aa6ac8b05402da7587e83e5c4ce09 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:23.473565 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651c775-ca80-4061-ae62-a0d5fc2a988c None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:23.507979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:23.571032 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651c775-ca80-4061-ae62-a0d5fc2a988c None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:23.602768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682cfc8-7502-418f-bd05-218259000d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:23.603332 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682cfc8-7502-418f-bd05-218259000d22 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:23.603608 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682cfc8-7502-418f-bd05-218259000d22 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:23.603878 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682cfc8-7502-418f-bd05-218259000d22 None admin] PATH_INFO: `/v3/users/cc44535a61884dfd8c56c49559e43165` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:23.616886 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5682cfc8-7502-418f-bd05-218259000d22 None admin] RBAC: Authorizing `identity:get_user(user_id=cc44535a61884dfd8c56c49559e43165)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:23.618307 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5682cfc8-7502-418f-bd05-218259000d22 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:23.619860 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 563/1114] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:36:23 2022] GET /identity/v3/users/cc44535a61884dfd8c56c49559e43165 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:23.626148 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a120b-80fe-4d36-8527-3ea2991cf24b None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:23.720628 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a120b-80fe-4d36-8527-3ea2991cf24b None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:23.745125 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:23.753373 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e10ed-06e4-4f5f-9301-367d47f03ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:23.753967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e10ed-06e4-4f5f-9301-367d47f03ca3 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:23.757845 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e10ed-06e4-4f5f-9301-367d47f03ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:23.757845 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e10ed-06e4-4f5f-9301-367d47f03ca3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:23.818278 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864e10ed-06e4-4f5f-9301-367d47f03ca3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:23.819704 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864e10ed-06e4-4f5f-9301-367d47f03ca3 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:23.914388 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 552/1115] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:36:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 00:36:23.920778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682cfc8-7502-418f-bd05-218259000d22 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:24.009227 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:24.016146 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682cfc8-7502-418f-bd05-218259000d22 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:24.049054 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b8910-a456-460c-92e3-691a96711e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:24.049668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b8910-a456-460c-92e3-691a96711e37 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:24.049943 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b8910-a456-460c-92e3-691a96711e37 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:24.050218 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b8910-a456-460c-92e3-691a96711e37 None admin] PATH_INFO: `/v3/projects/abf5cf909af24d2281230165a0bd8526` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:24.051303 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b8910-a456-460c-92e3-691a96711e37 None admin] RBAC: Authorizing `identity:get_project(project_id=abf5cf909af24d2281230165a0bd8526)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:24.052742 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b8910-a456-460c-92e3-691a96711e37 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:24.054324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 564/1116] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:36:23 2022] GET /identity/v3/projects/abf5cf909af24d2281230165a0bd8526 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:24.060410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e10ed-06e4-4f5f-9301-367d47f03ca3 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:24.086648 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:24.157653 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e10ed-06e4-4f5f-9301-367d47f03ca3 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:24.191213 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed65d8-4af1-407c-8936-c9d2cc3db55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:24.191751 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed65d8-4af1-407c-8936-c9d2cc3db55e None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:24.192029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed65d8-4af1-407c-8936-c9d2cc3db55e None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:24.192301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed65d8-4af1-407c-8936-c9d2cc3db55e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:24.193057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed65d8-4af1-407c-8936-c9d2cc3db55e None admin] RBAC: Adding query filter params (role.id=423aa6ac8b05402da7587e83e5c4ce09, scope.project.id=abf5cf909af24d2281230165a0bd8526, user.id=cc44535a61884dfd8c56c49559e43165) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:24.193373 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed65d8-4af1-407c-8936-c9d2cc3db55e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:24.194787 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed65d8-4af1-407c-8936-c9d2cc3db55e None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:24.199953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 553/1117] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:36:24 2022] GET /identity/v3/role_assignments?user.id=cc44535a61884dfd8c56c49559e43165&scope.project.id=abf5cf909af24d2281230165a0bd8526&role.id=423aa6ac8b05402da7587e83e5c4ce09 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:26.042142 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8a255-7f50-46e7-836b-a66bf82360fd None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:26.042706 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8a255-7f50-46e7-836b-a66bf82360fd None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:26.042706 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8a255-7f50-46e7-836b-a66bf82360fd None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:26.043997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 565/1118] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:36:26.049385 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ffde8-0eaa-4375-ae19-fa6dcf70a2c7 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:26.049781 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ffde8-0eaa-4375-ae19-fa6dcf70a2c7 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:26.049941 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ffde8-0eaa-4375-ae19-fa6dcf70a2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:26.068828 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-b71ffde8-0eaa-4375-ae19-fa6dcf70a2c7 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:26.070167 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-b71ffde8-0eaa-4375-ae19-fa6dcf70a2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:26.135372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 554/1119] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:36:26.412458 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8a255-7f50-46e7-836b-a66bf82360fd None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:26.510662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8a255-7f50-46e7-836b-a66bf82360fd None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:26.543751 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae70d1f-6e25-45d8-b1a7-28045b3d83f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:26.544385 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae70d1f-6e25-45d8-b1a7-28045b3d83f0 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:26.544689 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae70d1f-6e25-45d8-b1a7-28045b3d83f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:26.544959 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae70d1f-6e25-45d8-b1a7-28045b3d83f0 None admin] PATH_INFO: `/v3/roles/423aa6ac8b05402da7587e83e5c4ce09` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:26.546313 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae70d1f-6e25-45d8-b1a7-28045b3d83f0 None admin] RBAC: Authorizing `identity:get_role(role_id=423aa6ac8b05402da7587e83e5c4ce09)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:26.547659 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae70d1f-6e25-45d8-b1a7-28045b3d83f0 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:26.549007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 566/1120] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:36:26 2022] GET /identity/v3/roles/423aa6ac8b05402da7587e83e5c4ce09 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:26.554882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ffde8-0eaa-4375-ae19-fa6dcf70a2c7 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:26.575088 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:26.650654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ffde8-0eaa-4375-ae19-fa6dcf70a2c7 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:26.683222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da81ba7-e4c1-47ae-86b0-7b4ce8224f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:26.683783 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da81ba7-e4c1-47ae-86b0-7b4ce8224f13 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:26.684066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da81ba7-e4c1-47ae-86b0-7b4ce8224f13 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:26.684342 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da81ba7-e4c1-47ae-86b0-7b4ce8224f13 None admin] PATH_INFO: `/v3/users/cc44535a61884dfd8c56c49559e43165` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:26.686006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da81ba7-e4c1-47ae-86b0-7b4ce8224f13 None admin] RBAC: Authorizing `identity:get_user(user_id=cc44535a61884dfd8c56c49559e43165)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:26.687425 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da81ba7-e4c1-47ae-86b0-7b4ce8224f13 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:26.689012 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 555/1121] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:36:26 2022] GET /identity/v3/users/cc44535a61884dfd8c56c49559e43165 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:26.694800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae70d1f-6e25-45d8-b1a7-28045b3d83f0 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:26.724761 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:26.792119 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae70d1f-6e25-45d8-b1a7-28045b3d83f0 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:26.824252 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf9908-0a9f-4095-8e33-99649a061a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:26.824892 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf9908-0a9f-4095-8e33-99649a061a96 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:26.825169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf9908-0a9f-4095-8e33-99649a061a96 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:26.825478 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf9908-0a9f-4095-8e33-99649a061a96 None admin] PATH_INFO: `/v3/projects/abf5cf909af24d2281230165a0bd8526` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:26.827160 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf9908-0a9f-4095-8e33-99649a061a96 None admin] RBAC: Authorizing `identity:get_project(project_id=abf5cf909af24d2281230165a0bd8526)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:26.828598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf9908-0a9f-4095-8e33-99649a061a96 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:26.830191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 567/1122] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:36:26 2022] GET /identity/v3/projects/abf5cf909af24d2281230165a0bd8526 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:26.836147 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da81ba7-e4c1-47ae-86b0-7b4ce8224f13 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:26.921049 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:26.932813 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da81ba7-e4c1-47ae-86b0-7b4ce8224f13 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:26.966708 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1e96a-1b86-40be-9637-ab8b9292c792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:26.967299 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1e96a-1b86-40be-9637-ab8b9292c792 None admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:26.967580 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1e96a-1b86-40be-9637-ab8b9292c792 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:26.967855 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1e96a-1b86-40be-9637-ab8b9292c792 None admin] PATH_INFO: `/v3/projects/abf5cf909af24d2281230165a0bd8526/users/cc44535a61884dfd8c56c49559e43165/roles/423aa6ac8b05402da7587e83e5c4ce09` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:26.969997 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1e96a-1b86-40be-9637-ab8b9292c792 None admin] RBAC: Authorizing `identity:create_grant(project_id=abf5cf909af24d2281230165a0bd8526, user_id=cc44535a61884dfd8c56c49559e43165, role_id=423aa6ac8b05402da7587e83e5c4ce09)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:26.971420 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1e96a-1b86-40be-9637-ab8b9292c792 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:27.053402 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 556/1123] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Aug 26 00:36:26 2022] PUT /identity/v3/projects/abf5cf909af24d2281230165a0bd8526/users/cc44535a61884dfd8c56c49559e43165/roles/423aa6ac8b05402da7587e83e5c4ce09 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:36:28.952970 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e0d44-8be9-451e-809a-bb0754d8db6e None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:28.953507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e0d44-8be9-451e-809a-bb0754d8db6e None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:28.953507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e0d44-8be9-451e-809a-bb0754d8db6e None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:28.954856 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 568/1124] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:36:28.960078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361681a-6d2c-456d-b1bb-2815349156a5 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:28.960564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361681a-6d2c-456d-b1bb-2815349156a5 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:28.960673 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361681a-6d2c-456d-b1bb-2815349156a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:28.977987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-d361681a-6d2c-456d-b1bb-2815349156a5 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:28.979155 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-d361681a-6d2c-456d-b1bb-2815349156a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:29.047154 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 557/1125] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:36:29.319891 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e0d44-8be9-451e-809a-bb0754d8db6e None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:29.417964 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:29.422363 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e0d44-8be9-451e-809a-bb0754d8db6e None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:29.456069 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82cb17-146a-4d69-924e-424ade120f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:29.456732 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82cb17-146a-4d69-924e-424ade120f70 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:29.457017 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82cb17-146a-4d69-924e-424ade120f70 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:29.457295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82cb17-146a-4d69-924e-424ade120f70 None admin] PATH_INFO: `/v3/roles/423aa6ac8b05402da7587e83e5c4ce09` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:29.458686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82cb17-146a-4d69-924e-424ade120f70 None admin] RBAC: Authorizing `identity:get_role(role_id=423aa6ac8b05402da7587e83e5c4ce09)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:29.460183 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82cb17-146a-4d69-924e-424ade120f70 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:29.461523 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 569/1126] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:36:29 2022] GET /identity/v3/roles/423aa6ac8b05402da7587e83e5c4ce09 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:29.467767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361681a-6d2c-456d-b1bb-2815349156a5 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:29.542131 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:29.562065 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361681a-6d2c-456d-b1bb-2815349156a5 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:29.595586 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9055c5-e422-45a1-a16f-9b100c41adbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:29.596158 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9055c5-e422-45a1-a16f-9b100c41adbf None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:29.596431 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9055c5-e422-45a1-a16f-9b100c41adbf None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:29.596734 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9055c5-e422-45a1-a16f-9b100c41adbf None admin] PATH_INFO: `/v3/users/cc44535a61884dfd8c56c49559e43165` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:29.597842 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9055c5-e422-45a1-a16f-9b100c41adbf None admin] RBAC: Authorizing `identity:get_user(user_id=cc44535a61884dfd8c56c49559e43165)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:29.599258 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9055c5-e422-45a1-a16f-9b100c41adbf None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:29.600853 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 558/1127] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:36:29 2022] GET /identity/v3/users/cc44535a61884dfd8c56c49559e43165 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:36:29.606485 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82cb17-146a-4d69-924e-424ade120f70 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:29.704320 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82cb17-146a-4d69-924e-424ade120f70 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:29.737240 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace66df2-cb7c-418b-b12e-ba5bab9f6c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:29.737835 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace66df2-cb7c-418b-b12e-ba5bab9f6c00 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:29.738116 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace66df2-cb7c-418b-b12e-ba5bab9f6c00 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:29.738390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace66df2-cb7c-418b-b12e-ba5bab9f6c00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:29.771550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:29.804443 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace66df2-cb7c-418b-b12e-ba5bab9f6c00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:29.805907 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace66df2-cb7c-418b-b12e-ba5bab9f6c00 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:29.902087 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 570/1128] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:36:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 00:36:29.908184 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9055c5-e422-45a1-a16f-9b100c41adbf None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:30.005120 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9055c5-e422-45a1-a16f-9b100c41adbf None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:30.038691 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c4381-ade6-49ff-aab6-86eb21e28dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:30.039313 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c4381-ade6-49ff-aab6-86eb21e28dcf None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:30.039587 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c4381-ade6-49ff-aab6-86eb21e28dcf None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:30.039856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c4381-ade6-49ff-aab6-86eb21e28dcf None admin] PATH_INFO: `/v3/projects/abf5cf909af24d2281230165a0bd8526` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:30.041045 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c4381-ade6-49ff-aab6-86eb21e28dcf None admin] RBAC: Authorizing `identity:get_project(project_id=abf5cf909af24d2281230165a0bd8526)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:30.042463 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c4381-ade6-49ff-aab6-86eb21e28dcf None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:30.044070 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 559/1129] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:36:29 2022] GET /identity/v3/projects/abf5cf909af24d2281230165a0bd8526 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:30.050766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace66df2-cb7c-418b-b12e-ba5bab9f6c00 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:30.117292 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:30.149890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace66df2-cb7c-418b-b12e-ba5bab9f6c00 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:30.182359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314db5d8-c3d7-421e-8a87-e543c3c9aa21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:30.182896 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314db5d8-c3d7-421e-8a87-e543c3c9aa21 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:30.183177 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314db5d8-c3d7-421e-8a87-e543c3c9aa21 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:30.183451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314db5d8-c3d7-421e-8a87-e543c3c9aa21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:30.184163 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314db5d8-c3d7-421e-8a87-e543c3c9aa21 None admin] RBAC: Adding query filter params (role.id=423aa6ac8b05402da7587e83e5c4ce09, scope.project.id=abf5cf909af24d2281230165a0bd8526, user.id=cc44535a61884dfd8c56c49559e43165) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:30.184493 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314db5d8-c3d7-421e-8a87-e543c3c9aa21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:30.186599 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314db5d8-c3d7-421e-8a87-e543c3c9aa21 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:30.192141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 571/1130] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:36:30 2022] GET /identity/v3/role_assignments?user.id=cc44535a61884dfd8c56c49559e43165&scope.project.id=abf5cf909af24d2281230165a0bd8526&role.id=423aa6ac8b05402da7587e83e5c4ce09 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:32.105167 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751dec3-570e-4501-9b96-52bd2c519f82 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:32.105748 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751dec3-570e-4501-9b96-52bd2c519f82 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:32.105748 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751dec3-570e-4501-9b96-52bd2c519f82 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:32.107571 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 560/1131] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:36:32.112874 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb41d1-0b06-4614-bca4-d1838f327bea None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:32.113287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb41d1-0b06-4614-bca4-d1838f327bea None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:32.113420 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb41d1-0b06-4614-bca4-d1838f327bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:32.130968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-34cb41d1-0b06-4614-bca4-d1838f327bea None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:32.132149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-34cb41d1-0b06-4614-bca4-d1838f327bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:32.195804 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 572/1132] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:36:32.472311 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751dec3-570e-4501-9b96-52bd2c519f82 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:32.489014 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:32.570165 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751dec3-570e-4501-9b96-52bd2c519f82 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:32.602189 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7f578-d914-4e94-9692-542b0b3ebe6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:32.602805 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7f578-d914-4e94-9692-542b0b3ebe6e None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:32.603080 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7f578-d914-4e94-9692-542b0b3ebe6e None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:32.603349 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7f578-d914-4e94-9692-542b0b3ebe6e None admin] PATH_INFO: `/v3/domains/default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:32.604428 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae7f578-d914-4e94-9692-542b0b3ebe6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:32.605869 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae7f578-d914-4e94-9692-542b0b3ebe6e None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:32.607917 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 561/1133] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:36:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:32.613716 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb41d1-0b06-4614-bca4-d1838f327bea None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:32.670339 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:32.709707 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb41d1-0b06-4614-bca4-d1838f327bea None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:32.741253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57afc3-9f11-42d1-9bb2-cf3def0b12ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:32.741850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57afc3-9f11-42d1-9bb2-cf3def0b12ad None admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:32.742126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57afc3-9f11-42d1-9bb2-cf3def0b12ad None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:32.742395 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57afc3-9f11-42d1-9bb2-cf3def0b12ad None admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:32.743039 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57afc3-9f11-42d1-9bb2-cf3def0b12ad None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:32.744450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57afc3-9f11-42d1-9bb2-cf3def0b12ad None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:32.855101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 573/1134] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Aug 26 00:36:32 2022] POST /identity/v3/projects => generated 311 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:36:34.768171 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d7a5b-6368-4620-8c70-c90670e347a1 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:34.768831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d7a5b-6368-4620-8c70-c90670e347a1 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:34.768831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d7a5b-6368-4620-8c70-c90670e347a1 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:34.769980 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 562/1135] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:36:34.775090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586aa2cf-ac2a-4e16-afe3-088b4a503085 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:34.775467 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586aa2cf-ac2a-4e16-afe3-088b4a503085 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:34.775649 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586aa2cf-ac2a-4e16-afe3-088b4a503085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:34.793604 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-586aa2cf-ac2a-4e16-afe3-088b4a503085 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:34.795278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-586aa2cf-ac2a-4e16-afe3-088b4a503085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:34.861234 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 574/1136] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:36:35.132329 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d7a5b-6368-4620-8c70-c90670e347a1 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:35.211758 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:35.230164 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d7a5b-6368-4620-8c70-c90670e347a1 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:35.263810 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133d4361-4442-4b9d-b5e6-ed4ce7d68a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:35.264430 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d4361-4442-4b9d-b5e6-ed4ce7d68a6b None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:35.264738 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d4361-4442-4b9d-b5e6-ed4ce7d68a6b None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:35.265011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d4361-4442-4b9d-b5e6-ed4ce7d68a6b None admin] PATH_INFO: `/v3/domains/default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:35.266107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133d4361-4442-4b9d-b5e6-ed4ce7d68a6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:35.267539 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133d4361-4442-4b9d-b5e6-ed4ce7d68a6b None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:35.269125 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 563/1137] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:36:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:35.275051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586aa2cf-ac2a-4e16-afe3-088b4a503085 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:35.323145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:35.377862 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586aa2cf-ac2a-4e16-afe3-088b4a503085 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:35.412191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3224d2-aa8c-47d7-a18a-1bce64fc4a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:35.412759 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3224d2-aa8c-47d7-a18a-1bce64fc4a6e None admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:35.413040 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3224d2-aa8c-47d7-a18a-1bce64fc4a6e None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:35.413313 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3224d2-aa8c-47d7-a18a-1bce64fc4a6e None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:35.413973 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3224d2-aa8c-47d7-a18a-1bce64fc4a6e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:35.415408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3224d2-aa8c-47d7-a18a-1bce64fc4a6e None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:35.492296 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 575/1138] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Aug 26 00:36:35 2022] POST /identity/v3/users => generated 286 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:36:37.398440 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942e540-77fb-49d4-b0b4-a3084896901d None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:37.399007 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942e540-77fb-49d4-b0b4-a3084896901d None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:37.399007 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942e540-77fb-49d4-b0b4-a3084896901d None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:37.400326 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 564/1139] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:36:37.406264 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254a148-20a2-4f27-bd5a-f84ddb7be054 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:37.406714 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254a148-20a2-4f27-bd5a-f84ddb7be054 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:37.406808 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254a148-20a2-4f27-bd5a-f84ddb7be054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:37.425165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-6254a148-20a2-4f27-bd5a-f84ddb7be054 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:37.426356 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-6254a148-20a2-4f27-bd5a-f84ddb7be054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:37.492008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 576/1140] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:36:37.757697 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942e540-77fb-49d4-b0b4-a3084896901d None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:37.854792 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942e540-77fb-49d4-b0b4-a3084896901d None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:37.887272 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a83b21-9a98-4009-baa2-a3c2c9c68dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:37.887920 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a83b21-9a98-4009-baa2-a3c2c9c68dc9 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:37.888198 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a83b21-9a98-4009-baa2-a3c2c9c68dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:37.888494 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a83b21-9a98-4009-baa2-a3c2c9c68dc9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:37.898104 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a83b21-9a98-4009-baa2-a3c2c9c68dc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:36:37.902716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-74a83b21-9a98-4009-baa2-a3c2c9c68dc9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:36:37.902716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:37.902716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:37.902716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:37.902716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:37.902716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:37.902716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:37.902716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:37.902716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:37.902716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:37.902716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:37.902716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:37.902716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:36:37.902716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:36:37.902716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:36:37.902716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:36:37.904735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:36:37.904735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:36:37.904735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:36:37.904735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:36:37.904735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:37.904735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:37.904735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:37.904735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:37.904735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:37.904735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:37.904735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:37.904735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:37.904735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:37.904735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:37.904735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:37.904735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:37.906035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:37.906035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:37.906035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:37.906035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:37.906035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:36:37.906035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:36:37.906035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:36:37.906035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:36:37.906035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:36:37.906035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:36:37.906035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:36:37.906035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:36:37.906035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 565/1141] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:36:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:36:37.910455 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254a148-20a2-4f27-bd5a-f84ddb7be054 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:37.937174 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:38.007087 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254a148-20a2-4f27-bd5a-f84ddb7be054 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:38.040562 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3068d3-2ee5-4dea-85a8-bd8f4d9927b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:38.041114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3068d3-2ee5-4dea-85a8-bd8f4d9927b1 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:38.041394 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3068d3-2ee5-4dea-85a8-bd8f4d9927b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:38.041669 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3068d3-2ee5-4dea-85a8-bd8f4d9927b1 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:38.042324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3068d3-2ee5-4dea-85a8-bd8f4d9927b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:38.042623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3068d3-2ee5-4dea-85a8-bd8f4d9927b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:38.043971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3068d3-2ee5-4dea-85a8-bd8f4d9927b1 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:38.052410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 577/1142] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:36:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:38.058195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a83b21-9a98-4009-baa2-a3c2c9c68dc9 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:38.070662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:38.153348 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a83b21-9a98-4009-baa2-a3c2c9c68dc9 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:38.186485 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224b60a-ba12-4887-b9c9-02e5efc6c336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:38.187103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224b60a-ba12-4887-b9c9-02e5efc6c336 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:38.187384 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224b60a-ba12-4887-b9c9-02e5efc6c336 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:38.187658 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224b60a-ba12-4887-b9c9-02e5efc6c336 None admin] PATH_INFO: `/v3/users/dadcd07455a3454b8b80815e6ea60ee5` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:38.200066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6224b60a-ba12-4887-b9c9-02e5efc6c336 None admin] RBAC: Authorizing `identity:get_user(user_id=dadcd07455a3454b8b80815e6ea60ee5)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:38.201650 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6224b60a-ba12-4887-b9c9-02e5efc6c336 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:38.203257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 566/1143] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:36:38 2022] GET /identity/v3/users/dadcd07455a3454b8b80815e6ea60ee5 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:38.209344 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3068d3-2ee5-4dea-85a8-bd8f4d9927b1 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:38.297833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:38.306850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3068d3-2ee5-4dea-85a8-bd8f4d9927b1 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:38.339133 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ecfbb-9dcd-4f4d-afd0-611912f605d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:38.339733 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ecfbb-9dcd-4f4d-afd0-611912f605d0 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:38.340012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ecfbb-9dcd-4f4d-afd0-611912f605d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:38.340285 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ecfbb-9dcd-4f4d-afd0-611912f605d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:38.406314 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043ecfbb-9dcd-4f4d-afd0-611912f605d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:38.407776 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043ecfbb-9dcd-4f4d-afd0-611912f605d0 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:38.493191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:38.502997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 578/1144] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:36:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 00:36:38.509275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224b60a-ba12-4887-b9c9-02e5efc6c336 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:38.576036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:38.609895 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224b60a-ba12-4887-b9c9-02e5efc6c336 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:38.641906 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee8b45-932b-4a88-b2a9-8435e5433930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:38.642527 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee8b45-932b-4a88-b2a9-8435e5433930 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:38.642809 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee8b45-932b-4a88-b2a9-8435e5433930 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:38.643087 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee8b45-932b-4a88-b2a9-8435e5433930 None admin] PATH_INFO: `/v3/projects/c855ad53bd91426389683b8535475651` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:38.644193 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee8b45-932b-4a88-b2a9-8435e5433930 None admin] RBAC: Authorizing `identity:get_project(project_id=c855ad53bd91426389683b8535475651)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:38.645659 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee8b45-932b-4a88-b2a9-8435e5433930 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:38.647259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 567/1145] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:36:38 2022] GET /identity/v3/projects/c855ad53bd91426389683b8535475651 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:38.654148 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ecfbb-9dcd-4f4d-afd0-611912f605d0 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:38.748747 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ecfbb-9dcd-4f4d-afd0-611912f605d0 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:38.781496 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ffaa8-c909-4ff0-ae87-8c3bef371e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:38.782084 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ffaa8-c909-4ff0-ae87-8c3bef371e31 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:38.782359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ffaa8-c909-4ff0-ae87-8c3bef371e31 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:38.782633 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ffaa8-c909-4ff0-ae87-8c3bef371e31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:38.783339 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ffaa8-c909-4ff0-ae87-8c3bef371e31 None admin] RBAC: Adding query filter params (role.id=1a33e3c8cb8347feb9498595886cc621, scope.project.id=c855ad53bd91426389683b8535475651, user.id=dadcd07455a3454b8b80815e6ea60ee5) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:38.783641 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ffaa8-c909-4ff0-ae87-8c3bef371e31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:38.785076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ffaa8-c909-4ff0-ae87-8c3bef371e31 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:38.790200 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 579/1146] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:36:38 2022] GET /identity/v3/role_assignments?user.id=dadcd07455a3454b8b80815e6ea60ee5&scope.project.id=c855ad53bd91426389683b8535475651&role.id=1a33e3c8cb8347feb9498595886cc621 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:40.646192 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f90b3-365d-4251-9b32-8ed0c8f21c9d None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:40.646736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f90b3-365d-4251-9b32-8ed0c8f21c9d None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:40.646736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f90b3-365d-4251-9b32-8ed0c8f21c9d None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:40.648082 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 568/1147] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:36:40.653166 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4641f-62a7-418f-90d5-59f98e95c703 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:40.654373 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4641f-62a7-418f-90d5-59f98e95c703 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:40.654532 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4641f-62a7-418f-90d5-59f98e95c703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:40.672549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-64a4641f-62a7-418f-90d5-59f98e95c703 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:40.673717 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-64a4641f-62a7-418f-90d5-59f98e95c703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:40.721287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:40.738056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 580/1148] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:36:41.020811 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f90b3-365d-4251-9b32-8ed0c8f21c9d None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:41.120255 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f90b3-365d-4251-9b32-8ed0c8f21c9d None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:41.152844 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6bd35-e7c9-4862-be72-e6563e7351d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:41.154235 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6bd35-e7c9-4862-be72-e6563e7351d6 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:41.154516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6bd35-e7c9-4862-be72-e6563e7351d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:41.154785 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6bd35-e7c9-4862-be72-e6563e7351d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:41.157372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:41.165047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce6bd35-e7c9-4862-be72-e6563e7351d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:36:41.170293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-7ce6bd35-e7c9-4862-be72-e6563e7351d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:36:41.170293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:41.170293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:41.170293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:41.170293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:41.170293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:41.170293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:41.170293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:41.170293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:41.170293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:41.170293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:41.170293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:41.170293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:36:41.170293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:36:41.170293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:36:41.170293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:36:41.175291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:36:41.175291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:36:41.175291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:36:41.175291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:36:41.175291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:41.175291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:41.175291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:41.175291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:41.175291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:41.175291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:41.175291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:41.175291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:41.175291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:41.175291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:41.175291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:41.175291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:41.176407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:41.176407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:41.176407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:41.176407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:41.176407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:36:41.176407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:36:41.176407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:36:41.176407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:36:41.176407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:36:41.176407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:36:41.176407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:36:41.176407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:36:41.176407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 569/1149] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:36:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:36:41.183458 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4641f-62a7-418f-90d5-59f98e95c703 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:41.279510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4641f-62a7-418f-90d5-59f98e95c703 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:41.311139 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d4460-02d7-41e2-a37e-250b717d6c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:41.311732 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d4460-02d7-41e2-a37e-250b717d6c39 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:41.312007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d4460-02d7-41e2-a37e-250b717d6c39 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:41.312273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d4460-02d7-41e2-a37e-250b717d6c39 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:41.312949 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d4460-02d7-41e2-a37e-250b717d6c39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:41.313244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d4460-02d7-41e2-a37e-250b717d6c39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:41.314595 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d4460-02d7-41e2-a37e-250b717d6c39 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:41.322307 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 581/1150] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:36:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:41.328641 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6bd35-e7c9-4862-be72-e6563e7351d6 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:41.426585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6bd35-e7c9-4862-be72-e6563e7351d6 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:41.459313 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc31703-9ceb-4a5f-9444-2f038fc0d572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:41.459937 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc31703-9ceb-4a5f-9444-2f038fc0d572 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:41.460213 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc31703-9ceb-4a5f-9444-2f038fc0d572 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:41.460502 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc31703-9ceb-4a5f-9444-2f038fc0d572 None admin] PATH_INFO: `/v3/users/dadcd07455a3454b8b80815e6ea60ee5` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:41.461614 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc31703-9ceb-4a5f-9444-2f038fc0d572 None admin] RBAC: Authorizing `identity:get_user(user_id=dadcd07455a3454b8b80815e6ea60ee5)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:41.463031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc31703-9ceb-4a5f-9444-2f038fc0d572 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:41.464696 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 570/1151] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:36:41 2022] GET /identity/v3/users/dadcd07455a3454b8b80815e6ea60ee5 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:41.471124 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d4460-02d7-41e2-a37e-250b717d6c39 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:41.519270 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:41.570884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d4460-02d7-41e2-a37e-250b717d6c39 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:41.603372 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18032f2-f985-415a-a876-0653ad7d3018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:41.603994 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18032f2-f985-415a-a876-0653ad7d3018 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:41.604272 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18032f2-f985-415a-a876-0653ad7d3018 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:41.604576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18032f2-f985-415a-a876-0653ad7d3018 None admin] PATH_INFO: `/v3/projects/c855ad53bd91426389683b8535475651` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:41.605668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18032f2-f985-415a-a876-0653ad7d3018 None admin] RBAC: Authorizing `identity:get_project(project_id=c855ad53bd91426389683b8535475651)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:41.607585 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18032f2-f985-415a-a876-0653ad7d3018 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:41.609226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 582/1152] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:36:41 2022] GET /identity/v3/projects/c855ad53bd91426389683b8535475651 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:41.615198 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc31703-9ceb-4a5f-9444-2f038fc0d572 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:41.675246 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:41.712820 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc31703-9ceb-4a5f-9444-2f038fc0d572 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:41.745369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a56cba-67b1-4896-825e-1caf3139e217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:41.746015 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a56cba-67b1-4896-825e-1caf3139e217 None admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:41.746295 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a56cba-67b1-4896-825e-1caf3139e217 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:41.746568 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a56cba-67b1-4896-825e-1caf3139e217 None admin] PATH_INFO: `/v3/projects/c855ad53bd91426389683b8535475651/users/dadcd07455a3454b8b80815e6ea60ee5/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:41.748464 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a56cba-67b1-4896-825e-1caf3139e217 None admin] RBAC: Authorizing `identity:create_grant(project_id=c855ad53bd91426389683b8535475651, user_id=dadcd07455a3454b8b80815e6ea60ee5, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:41.749918 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a56cba-67b1-4896-825e-1caf3139e217 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:41.813135 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 571/1153] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Aug 26 00:36:41 2022] PUT /identity/v3/projects/c855ad53bd91426389683b8535475651/users/dadcd07455a3454b8b80815e6ea60ee5/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:36:43.688918 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e1d14-26cb-40ec-980b-9302e077c1f8 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:43.689419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e1d14-26cb-40ec-980b-9302e077c1f8 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:43.689419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e1d14-26cb-40ec-980b-9302e077c1f8 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:43.690728 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 583/1154] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:36:43.695916 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bb24b-5a91-4ffa-b40b-8fc3dd5af2bb None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:43.696347 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bb24b-5a91-4ffa-b40b-8fc3dd5af2bb None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:43.696454 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bb24b-5a91-4ffa-b40b-8fc3dd5af2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:43.713897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-fa2bb24b-5a91-4ffa-b40b-8fc3dd5af2bb None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:43.715053 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-fa2bb24b-5a91-4ffa-b40b-8fc3dd5af2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:43.777793 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 572/1155] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:36:44.041887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e1d14-26cb-40ec-980b-9302e077c1f8 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:44.142508 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e1d14-26cb-40ec-980b-9302e077c1f8 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:44.157256 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:44.178305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a2e2a-d0c7-4760-8874-ce9d0d221370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:44.178943 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a2e2a-d0c7-4760-8874-ce9d0d221370 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:44.179226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a2e2a-d0c7-4760-8874-ce9d0d221370 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:44.179498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a2e2a-d0c7-4760-8874-ce9d0d221370 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:44.190115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2a2e2a-d0c7-4760-8874-ce9d0d221370 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:36:44.194818 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-cc2a2e2a-d0c7-4760-8874-ce9d0d221370 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:36:44.194818 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:44.194818 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:44.194818 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:44.194818 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:44.194818 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:44.194818 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:44.194818 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:44.194818 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:44.194818 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:44.194818 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:44.194818 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:44.194818 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:36:44.194818 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:36:44.194818 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:36:44.194818 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:36:44.195989 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:36:44.195989 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:36:44.195989 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:36:44.195989 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:36:44.195989 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:44.195989 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:44.195989 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:44.195989 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:44.195989 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:44.195989 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:44.195989 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:44.195989 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:44.195989 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:44.195989 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:44.195989 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:44.195989 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:44.196913 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:44.196913 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:44.196913 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:44.196913 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:44.196913 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:36:44.196913 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:36:44.196913 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:36:44.196913 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:36:44.196913 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:36:44.196913 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:36:44.196913 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:36:44.196913 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:36:44.196913 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 584/1156] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:36:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:36:44.202033 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2bb24b-5a91-4ffa-b40b-8fc3dd5af2bb None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:44.255263 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:44.299011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2bb24b-5a91-4ffa-b40b-8fc3dd5af2bb None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:44.331557 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cad887-d5b0-47bc-946d-cf321ca2a163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:44.332151 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cad887-d5b0-47bc-946d-cf321ca2a163 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:44.332431 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cad887-d5b0-47bc-946d-cf321ca2a163 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:44.332735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cad887-d5b0-47bc-946d-cf321ca2a163 None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:44.333393 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cad887-d5b0-47bc-946d-cf321ca2a163 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:44.333695 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cad887-d5b0-47bc-946d-cf321ca2a163 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:44.335058 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cad887-d5b0-47bc-946d-cf321ca2a163 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:44.344349 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 573/1157] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:36:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:44.350205 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a2e2a-d0c7-4760-8874-ce9d0d221370 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:44.446467 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a2e2a-d0c7-4760-8874-ce9d0d221370 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:44.478870 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d5c439-d9ef-44c2-bbaf-9b8503d3a8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:44.479491 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5c439-d9ef-44c2-bbaf-9b8503d3a8ea None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:44.479766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5c439-d9ef-44c2-bbaf-9b8503d3a8ea None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:44.480035 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5c439-d9ef-44c2-bbaf-9b8503d3a8ea None admin] PATH_INFO: `/v3/users/dadcd07455a3454b8b80815e6ea60ee5` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:44.481143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d5c439-d9ef-44c2-bbaf-9b8503d3a8ea None admin] RBAC: Authorizing `identity:get_user(user_id=dadcd07455a3454b8b80815e6ea60ee5)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:44.483530 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d5c439-d9ef-44c2-bbaf-9b8503d3a8ea None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:44.485143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 585/1158] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:36:44 2022] GET /identity/v3/users/dadcd07455a3454b8b80815e6ea60ee5 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:44.491218 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cad887-d5b0-47bc-946d-cf321ca2a163 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:44.589461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cad887-d5b0-47bc-946d-cf321ca2a163 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:44.604671 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:44.623394 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3da2cf-cb75-4a86-92dc-47ca03da4bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:44.624002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3da2cf-cb75-4a86-92dc-47ca03da4bc2 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:44.624279 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3da2cf-cb75-4a86-92dc-47ca03da4bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:44.624585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3da2cf-cb75-4a86-92dc-47ca03da4bc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:44.689533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3da2cf-cb75-4a86-92dc-47ca03da4bc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:44.690966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3da2cf-cb75-4a86-92dc-47ca03da4bc2 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:44.785106 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 574/1159] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:36:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 00:36:44.791011 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d5c439-d9ef-44c2-bbaf-9b8503d3a8ea None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:44.833504 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:44.889863 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d5c439-d9ef-44c2-bbaf-9b8503d3a8ea None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:44.923327 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d03827-58eb-4153-8887-8d9beaa53bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:44.923947 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d03827-58eb-4153-8887-8d9beaa53bfc None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:44.924229 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d03827-58eb-4153-8887-8d9beaa53bfc None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:44.924528 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d03827-58eb-4153-8887-8d9beaa53bfc None admin] PATH_INFO: `/v3/projects/c855ad53bd91426389683b8535475651` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:44.925640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d03827-58eb-4153-8887-8d9beaa53bfc None admin] RBAC: Authorizing `identity:get_project(project_id=c855ad53bd91426389683b8535475651)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:44.927056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d03827-58eb-4153-8887-8d9beaa53bfc None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:44.928685 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 586/1160] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:36:44 2022] GET /identity/v3/projects/c855ad53bd91426389683b8535475651 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:44.935186 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3da2cf-cb75-4a86-92dc-47ca03da4bc2 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:44.952634 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:45.031128 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3da2cf-cb75-4a86-92dc-47ca03da4bc2 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:45.063134 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6c6b5-465c-4105-b8b3-85dcdbfba658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:45.063724 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6c6b5-465c-4105-b8b3-85dcdbfba658 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:45.064004 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6c6b5-465c-4105-b8b3-85dcdbfba658 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:45.064275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6c6b5-465c-4105-b8b3-85dcdbfba658 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:45.065021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e6c6b5-465c-4105-b8b3-85dcdbfba658 None admin] RBAC: Adding query filter params (role.id=1a33e3c8cb8347feb9498595886cc621, scope.project.id=c855ad53bd91426389683b8535475651, user.id=dadcd07455a3454b8b80815e6ea60ee5) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:45.065325 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e6c6b5-465c-4105-b8b3-85dcdbfba658 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:45.066734 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e6c6b5-465c-4105-b8b3-85dcdbfba658 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:45.072381 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 575/1161] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:36:44 2022] GET /identity/v3/role_assignments?user.id=dadcd07455a3454b8b80815e6ea60ee5&scope.project.id=c855ad53bd91426389683b8535475651&role.id=1a33e3c8cb8347feb9498595886cc621 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:46.995106 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339e1d4-0a8d-41af-90a7-099808b110bc None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:46.995644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339e1d4-0a8d-41af-90a7-099808b110bc None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:46.995644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339e1d4-0a8d-41af-90a7-099808b110bc None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:46.997675 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 587/1162] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:36:47.002777 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88774a0c-8dec-4900-9557-86985567068b None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:47.003331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88774a0c-8dec-4900-9557-86985567068b None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:47.003331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88774a0c-8dec-4900-9557-86985567068b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:47.021255 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-88774a0c-8dec-4900-9557-86985567068b None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:47.022442 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-88774a0c-8dec-4900-9557-86985567068b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:47.085551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 576/1163] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:36:47.356209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339e1d4-0a8d-41af-90a7-099808b110bc None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:47.456859 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339e1d4-0a8d-41af-90a7-099808b110bc None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:47.467257 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:47.490276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f8290-fcda-4d1b-9113-2bcab33cc558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:47.490885 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f8290-fcda-4d1b-9113-2bcab33cc558 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:47.491167 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f8290-fcda-4d1b-9113-2bcab33cc558 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:47.491443 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f8290-fcda-4d1b-9113-2bcab33cc558 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:47.557358 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f8290-fcda-4d1b-9113-2bcab33cc558 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:47.558959 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f8290-fcda-4d1b-9113-2bcab33cc558 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:47.654754 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 588/1164] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:36:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 00:36:47.660635 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88774a0c-8dec-4900-9557-86985567068b None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:47.672356 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:47.754959 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88774a0c-8dec-4900-9557-86985567068b None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:47.787664 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2620ec9d-eb5b-4598-a518-58cf5be5b6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:47.788273 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620ec9d-eb5b-4598-a518-58cf5be5b6ac None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:47.788573 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620ec9d-eb5b-4598-a518-58cf5be5b6ac None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:47.788841 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620ec9d-eb5b-4598-a518-58cf5be5b6ac None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:47.795190 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2620ec9d-eb5b-4598-a518-58cf5be5b6ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:47.796653 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2620ec9d-eb5b-4598-a518-58cf5be5b6ac None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:47.807379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-2620ec9d-eb5b-4598-a518-58cf5be5b6ac None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 26 00:36:47.807379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:47.807379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 26 00:36:47.807379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 26 00:36:47.807379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:36:47.807379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:36:47.807379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:36:47.807379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:36:47.807379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 26 00:36:47.807379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:36:47.807379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:36:47.807379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:36:47.807379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:47.807379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:47.807379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:47.807379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:47.807379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:47.808700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:47.808700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:47.808700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:47.808700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:47.808700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:47.808700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:47.808700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:36:47.808700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:36:47.808700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:36:47.808700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:36:47.808700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:47.808700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:47.808700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:47.808700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:47.808700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:47.808700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:47.810461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:47.810461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:47.810461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:47.810461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:47.810461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:47.810461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:47.810461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:47.810461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:47.810461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:47.810461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:47.810461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 26 00:36:47.810461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:36:47.810461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 26 00:36:47.810461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:36:47.810461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 577/1165] 10.222.0.59 () {56 vars in 1117 bytes} [Fri Aug 26 00:36:47 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:36:47.814873 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f8290-fcda-4d1b-9113-2bcab33cc558 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:47.827028 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:47.911787 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f8290-fcda-4d1b-9113-2bcab33cc558 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:47.944632 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab34b3d-e17b-4231-8d96-1794ecfb429b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:47.945245 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab34b3d-e17b-4231-8d96-1794ecfb429b None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:47.945530 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab34b3d-e17b-4231-8d96-1794ecfb429b None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:47.945802 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab34b3d-e17b-4231-8d96-1794ecfb429b None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:47.946445 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab34b3d-e17b-4231-8d96-1794ecfb429b None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:47.946752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab34b3d-e17b-4231-8d96-1794ecfb429b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:47.948118 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab34b3d-e17b-4231-8d96-1794ecfb429b None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:47.955758 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 589/1166] 10.222.0.59 () {56 vars in 1115 bytes} [Fri Aug 26 00:36:47 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:49.814654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cca9b5-58fc-4e54-bfd4-613804795356 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:49.815192 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cca9b5-58fc-4e54-bfd4-613804795356 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:49.815288 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cca9b5-58fc-4e54-bfd4-613804795356 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:49.816616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 578/1167] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:36:49.821634 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13be68-8cd0-4e7d-b667-4eecd4db3cda None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:49.822015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13be68-8cd0-4e7d-b667-4eecd4db3cda None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:49.822183 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13be68-8cd0-4e7d-b667-4eecd4db3cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:49.839724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-0a13be68-8cd0-4e7d-b667-4eecd4db3cda None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:49.840944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-0a13be68-8cd0-4e7d-b667-4eecd4db3cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:49.909214 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 590/1168] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 00:36:50.177985 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cca9b5-58fc-4e54-bfd4-613804795356 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:50.246552 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:50.276161 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cca9b5-58fc-4e54-bfd4-613804795356 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:50.310145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b10242-aba3-44bf-89d7-88770f563e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:50.310740 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b10242-aba3-44bf-89d7-88770f563e0c None admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:50.311016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b10242-aba3-44bf-89d7-88770f563e0c None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:50.311284 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b10242-aba3-44bf-89d7-88770f563e0c None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:50.311940 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b10242-aba3-44bf-89d7-88770f563e0c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:50.313316 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b10242-aba3-44bf-89d7-88770f563e0c None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:50.389542 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 579/1169] 10.222.0.59 () {60 vars in 1138 bytes} [Fri Aug 26 00:36:50 2022] POST /identity/v3/domains => generated 267 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:36:52.358787 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcae662-8869-4471-bd17-5e503571f96f None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:52.359324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcae662-8869-4471-bd17-5e503571f96f None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:52.359324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcae662-8869-4471-bd17-5e503571f96f None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:52.360703 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 591/1170] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:36:52.365772 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720a754-d40c-4369-a0a1-9c94c098de74 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:52.366200 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720a754-d40c-4369-a0a1-9c94c098de74 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:52.366320 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720a754-d40c-4369-a0a1-9c94c098de74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:52.383643 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-f720a754-d40c-4369-a0a1-9c94c098de74 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:52.384867 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-f720a754-d40c-4369-a0a1-9c94c098de74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:52.450400 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 580/1171] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:36:52.723618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcae662-8869-4471-bd17-5e503571f96f None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:52.727404 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:52.824521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcae662-8869-4471-bd17-5e503571f96f None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:52.857881 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa71d8-a89c-4230-8706-0a890cc7f8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:52.858501 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa71d8-a89c-4230-8706-0a890cc7f8d8 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:52.858781 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa71d8-a89c-4230-8706-0a890cc7f8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:52.859053 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa71d8-a89c-4230-8706-0a890cc7f8d8 None admin] PATH_INFO: `/v3/domains/1a84a1f17d584650ace3425c2f0f265b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:52.869146 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aa71d8-a89c-4230-8706-0a890cc7f8d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1a84a1f17d584650ace3425c2f0f265b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:52.870579 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aa71d8-a89c-4230-8706-0a890cc7f8d8 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:52.872676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 592/1172] 10.222.0.59 () {56 vars in 1183 bytes} [Fri Aug 26 00:36:52 2022] GET /identity/v3/domains/1a84a1f17d584650ace3425c2f0f265b => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:52.878524 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f720a754-d40c-4369-a0a1-9c94c098de74 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:52.934174 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:52.972633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f720a754-d40c-4369-a0a1-9c94c098de74 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:53.004851 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c7a3b-0a7c-4d15-adaf-35614250b29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:53.005388 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c7a3b-0a7c-4d15-adaf-35614250b29e None admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:53.005666 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c7a3b-0a7c-4d15-adaf-35614250b29e None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:53.005939 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c7a3b-0a7c-4d15-adaf-35614250b29e None admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:53.006575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c7a3b-0a7c-4d15-adaf-35614250b29e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:53.007996 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c7a3b-0a7c-4d15-adaf-35614250b29e None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:53.123028 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 581/1173] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Aug 26 00:36:52 2022] POST /identity/v3/projects => generated 361 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:36:55.053321 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42147e55-994c-4569-88e3-d5175967a214 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:55.053919 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42147e55-994c-4569-88e3-d5175967a214 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:55.053919 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42147e55-994c-4569-88e3-d5175967a214 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:55.055176 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 593/1174] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:36:55.061084 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2800a64-a03e-435e-a2a6-043513d1d0cc None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:55.061492 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2800a64-a03e-435e-a2a6-043513d1d0cc None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:55.061639 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2800a64-a03e-435e-a2a6-043513d1d0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:55.079449 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-b2800a64-a03e-435e-a2a6-043513d1d0cc None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:55.080667 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-b2800a64-a03e-435e-a2a6-043513d1d0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:55.144830 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 582/1175] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:36:55.418709 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42147e55-994c-4569-88e3-d5175967a214 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:55.482371 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:55.519210 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42147e55-994c-4569-88e3-d5175967a214 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:55.551377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bad26c-7938-4002-8d7a-700e4721073f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:55.551992 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bad26c-7938-4002-8d7a-700e4721073f None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:55.552268 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bad26c-7938-4002-8d7a-700e4721073f None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:55.552564 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bad26c-7938-4002-8d7a-700e4721073f None admin] PATH_INFO: `/v3/domains/1a84a1f17d584650ace3425c2f0f265b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:55.553679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bad26c-7938-4002-8d7a-700e4721073f None admin] RBAC: Authorizing `identity:get_domain(domain_id=1a84a1f17d584650ace3425c2f0f265b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:55.555093 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bad26c-7938-4002-8d7a-700e4721073f None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:55.556671 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 594/1176] 10.222.0.59 () {56 vars in 1183 bytes} [Fri Aug 26 00:36:55 2022] GET /identity/v3/domains/1a84a1f17d584650ace3425c2f0f265b => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:55.563048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2800a64-a03e-435e-a2a6-043513d1d0cc None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:55.604003 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:55.658818 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2800a64-a03e-435e-a2a6-043513d1d0cc None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:55.692697 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb23a981-f594-4624-9039-2abeb35be574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:55.693239 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23a981-f594-4624-9039-2abeb35be574 None admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:55.693649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23a981-f594-4624-9039-2abeb35be574 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:55.693925 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23a981-f594-4624-9039-2abeb35be574 None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:55.694560 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb23a981-f594-4624-9039-2abeb35be574 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:55.695988 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb23a981-f594-4624-9039-2abeb35be574 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:55.773030 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 583/1177] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Aug 26 00:36:55 2022] POST /identity/v3/users => generated 311 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:36:57.708734 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae46650-1183-47f8-aeb5-cbe871b323d8 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:57.709251 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae46650-1183-47f8-aeb5-cbe871b323d8 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:57.709251 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae46650-1183-47f8-aeb5-cbe871b323d8 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:57.710547 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 595/1178] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:36:57.716164 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e6f2f-4f6c-43f7-8aba-19365110f375 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:57.716579 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e6f2f-4f6c-43f7-8aba-19365110f375 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:57.716749 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e6f2f-4f6c-43f7-8aba-19365110f375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:57.734517 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-ec0e6f2f-4f6c-43f7-8aba-19365110f375 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:36:57.735681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-ec0e6f2f-4f6c-43f7-8aba-19365110f375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:36:57.798455 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 584/1179] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:36:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:36:58.062351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae46650-1183-47f8-aeb5-cbe871b323d8 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:58.163336 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae46650-1183-47f8-aeb5-cbe871b323d8 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:58.191893 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:58.196558 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3257f-ace1-43fc-a3cb-0d471e83959a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:58.197182 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3257f-ace1-43fc-a3cb-0d471e83959a None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:58.197489 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3257f-ace1-43fc-a3cb-0d471e83959a None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:58.197764 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3257f-ace1-43fc-a3cb-0d471e83959a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:58.208104 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3257f-ace1-43fc-a3cb-0d471e83959a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:36:58.212979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-57f3257f-ace1-43fc-a3cb-0d471e83959a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:36:58.212979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:36:58.212979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:36:58.212979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:36:58.212979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:36:58.212979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:36:58.212979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:36:58.212979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:36:58.212979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:36:58.212979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:36:58.212979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:36:58.212979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:36:58.212979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:36:58.212979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:36:58.212979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:36:58.212979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:36:58.214192 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:36:58.214192 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:36:58.214192 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:36:58.214192 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:36:58.214192 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:36:58.214192 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:36:58.214192 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:36:58.214192 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:36:58.214192 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:36:58.214192 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:36:58.214192 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:36:58.214192 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:36:58.214192 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:36:58.214192 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:36:58.214192 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:36:58.214192 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:36:58.215301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:36:58.215301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:36:58.215301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:36:58.215301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:36:58.215301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:36:58.215301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:36:58.215301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:36:58.215301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:36:58.215301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:36:58.215301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:36:58.215301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:36:58.215301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:36:58.215301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 596/1180] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:36:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:36:58.219902 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e6f2f-4f6c-43f7-8aba-19365110f375 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:58.246406 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:58.314984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e6f2f-4f6c-43f7-8aba-19365110f375 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:58.353088 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d429db-7c34-47a7-9d36-ee6ce09c468a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:58.353697 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d429db-7c34-47a7-9d36-ee6ce09c468a None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:58.353995 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d429db-7c34-47a7-9d36-ee6ce09c468a None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:58.354249 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d429db-7c34-47a7-9d36-ee6ce09c468a None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:58.354909 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d429db-7c34-47a7-9d36-ee6ce09c468a None admin] RBAC: Adding query filter params (name=admin) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:58.355212 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d429db-7c34-47a7-9d36-ee6ce09c468a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:58.356595 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d429db-7c34-47a7-9d36-ee6ce09c468a None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:58.364313 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 585/1181] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:36:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:58.370057 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3257f-ace1-43fc-a3cb-0d471e83959a None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:58.467537 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3257f-ace1-43fc-a3cb-0d471e83959a None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:58.501226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451cf576-624b-415c-9895-4925f0272dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:58.501851 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cf576-624b-415c-9895-4925f0272dfa None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:58.502135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cf576-624b-415c-9895-4925f0272dfa None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:58.502410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cf576-624b-415c-9895-4925f0272dfa None admin] PATH_INFO: `/v3/users/2c89a3c49dcc40ceb3daf5d08853d995` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:58.516272 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451cf576-624b-415c-9895-4925f0272dfa None admin] RBAC: Authorizing `identity:get_user(user_id=2c89a3c49dcc40ceb3daf5d08853d995)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:58.517760 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451cf576-624b-415c-9895-4925f0272dfa None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:58.519363 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 597/1182] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:36:58 2022] GET /identity/v3/users/2c89a3c49dcc40ceb3daf5d08853d995 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:36:58.525308 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d429db-7c34-47a7-9d36-ee6ce09c468a None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:58.613207 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:58.622730 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d429db-7c34-47a7-9d36-ee6ce09c468a None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:58.654822 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e2058-2de6-4725-b648-52e54bddc2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:58.655414 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e2058-2de6-4725-b648-52e54bddc2de None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:58.655692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e2058-2de6-4725-b648-52e54bddc2de None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:58.655961 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e2058-2de6-4725-b648-52e54bddc2de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:58.719073 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e2058-2de6-4725-b648-52e54bddc2de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:58.720550 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e2058-2de6-4725-b648-52e54bddc2de None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:58.805043 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:58.815105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 586/1183] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:36:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 00:36:58.821049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451cf576-624b-415c-9895-4925f0272dfa None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:58.870234 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:36:58.921273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451cf576-624b-415c-9895-4925f0272dfa None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:58.953267 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc19c08-21a2-4694-8722-9ccc675756bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:58.953888 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc19c08-21a2-4694-8722-9ccc675756bc None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:58.954167 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc19c08-21a2-4694-8722-9ccc675756bc None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:58.954437 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc19c08-21a2-4694-8722-9ccc675756bc None admin] PATH_INFO: `/v3/projects/797f83f14e0642368d4d8f1fbd524b1c` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:58.955528 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc19c08-21a2-4694-8722-9ccc675756bc None admin] RBAC: Authorizing `identity:get_project(project_id=797f83f14e0642368d4d8f1fbd524b1c)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:58.956980 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc19c08-21a2-4694-8722-9ccc675756bc None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:58.958568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 598/1184] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:36:58 2022] GET /identity/v3/projects/797f83f14e0642368d4d8f1fbd524b1c => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:36:58.964635 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e2058-2de6-4725-b648-52e54bddc2de None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:36:59.060156 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e2058-2de6-4725-b648-52e54bddc2de None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:36:59.092552 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a958f2-6142-4dc2-a52c-613ed3ff5797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:36:59.093144 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a958f2-6142-4dc2-a52c-613ed3ff5797 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:36:59.093424 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a958f2-6142-4dc2-a52c-613ed3ff5797 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:36:59.093697 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a958f2-6142-4dc2-a52c-613ed3ff5797 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:36:59.094412 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a958f2-6142-4dc2-a52c-613ed3ff5797 None admin] RBAC: Adding query filter params (role.id=1a33e3c8cb8347feb9498595886cc621, scope.project.id=797f83f14e0642368d4d8f1fbd524b1c, user.id=2c89a3c49dcc40ceb3daf5d08853d995) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:36:59.094720 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a958f2-6142-4dc2-a52c-613ed3ff5797 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:36:59.096143 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a958f2-6142-4dc2-a52c-613ed3ff5797 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:36:59.101317 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 587/1185] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:36:58 2022] GET /identity/v3/role_assignments?user.id=2c89a3c49dcc40ceb3daf5d08853d995&scope.project.id=797f83f14e0642368d4d8f1fbd524b1c&role.id=1a33e3c8cb8347feb9498595886cc621 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:37:01.018706 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881f903-b596-408e-b9ee-9f37edfbc682 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:37:01.019240 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881f903-b596-408e-b9ee-9f37edfbc682 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:37:01.019240 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881f903-b596-408e-b9ee-9f37edfbc682 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:37:01.020624 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 599/1186] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:37:01.025764 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003bbc7-5672-4549-9591-ca221770bdb0 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:37:01.026204 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003bbc7-5672-4549-9591-ca221770bdb0 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:37:01.026306 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003bbc7-5672-4549-9591-ca221770bdb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:37:01.045248 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-6003bbc7-5672-4549-9591-ca221770bdb0 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:37:01.046418 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-6003bbc7-5672-4549-9591-ca221770bdb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:37:01.093369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:37:01.109620 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 588/1187] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:37:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:37:01.377520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881f903-b596-408e-b9ee-9f37edfbc682 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:37:01.477440 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881f903-b596-408e-b9ee-9f37edfbc682 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:37:01.494564 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:37:01.510723 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215485d4-13c4-495f-8595-201e8d9dfa3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:37:01.511348 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215485d4-13c4-495f-8595-201e8d9dfa3e None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:37:01.511627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215485d4-13c4-495f-8595-201e8d9dfa3e None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:37:01.511901 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215485d4-13c4-495f-8595-201e8d9dfa3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:37:01.522683 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215485d4-13c4-495f-8595-201e8d9dfa3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:37:01.527369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-215485d4-13c4-495f-8595-201e8d9dfa3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:37:01.527369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:37:01.527369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:37:01.527369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:37:01.527369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:37:01.527369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:37:01.527369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:37:01.527369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:37:01.527369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:37:01.527369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:37:01.527369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:37:01.527369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:37:01.527369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:37:01.527369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:37:01.527369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:37:01.527369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:37:01.532542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:37:01.532542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:37:01.532542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:37:01.532542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:37:01.532542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:37:01.532542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:37:01.532542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:37:01.532542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:37:01.532542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:37:01.532542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:37:01.532542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:37:01.532542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:37:01.532542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:37:01.532542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:37:01.532542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:37:01.532542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:37:01.533756 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:37:01.533756 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:37:01.533756 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:37:01.533756 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:37:01.533756 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:37:01.533756 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:37:01.533756 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:37:01.533756 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:37:01.533756 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:37:01.533756 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:37:01.533756 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:37:01.533756 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:37:01.533756 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 600/1188] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:37:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:37:01.535335 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003bbc7-5672-4549-9591-ca221770bdb0 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:37:01.627775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003bbc7-5672-4549-9591-ca221770bdb0 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:37:01.660409 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c304d-9fae-4eba-9a59-b9edbe5f801d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:37:01.661030 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c304d-9fae-4eba-9a59-b9edbe5f801d None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:37:01.661306 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c304d-9fae-4eba-9a59-b9edbe5f801d None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:37:01.661574 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c304d-9fae-4eba-9a59-b9edbe5f801d None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:37:01.662222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c304d-9fae-4eba-9a59-b9edbe5f801d None admin] RBAC: Adding query filter params (name=admin) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:37:01.662520 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c304d-9fae-4eba-9a59-b9edbe5f801d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:37:01.663866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c304d-9fae-4eba-9a59-b9edbe5f801d None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:37:01.671596 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 589/1189] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:37:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:37:01.677315 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215485d4-13c4-495f-8595-201e8d9dfa3e None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:37:01.773061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215485d4-13c4-495f-8595-201e8d9dfa3e None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:37:01.805599 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dafc0aa-fed2-443b-a3fe-e25da807e020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:37:01.806207 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafc0aa-fed2-443b-a3fe-e25da807e020 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:37:01.806487 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafc0aa-fed2-443b-a3fe-e25da807e020 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:37:01.806759 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafc0aa-fed2-443b-a3fe-e25da807e020 None admin] PATH_INFO: `/v3/users/2c89a3c49dcc40ceb3daf5d08853d995` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:37:01.807864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dafc0aa-fed2-443b-a3fe-e25da807e020 None admin] RBAC: Authorizing `identity:get_user(user_id=2c89a3c49dcc40ceb3daf5d08853d995)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:37:01.809332 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dafc0aa-fed2-443b-a3fe-e25da807e020 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:37:01.810864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 601/1190] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:37:01 2022] GET /identity/v3/users/2c89a3c49dcc40ceb3daf5d08853d995 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:37:01.816800 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c304d-9fae-4eba-9a59-b9edbe5f801d None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:37:01.861687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:37:01.911985 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c304d-9fae-4eba-9a59-b9edbe5f801d None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:37:01.943267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a77d90-c3bd-4d1f-bd89-9c7de5ff875a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:37:01.943842 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a77d90-c3bd-4d1f-bd89-9c7de5ff875a None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:37:01.944117 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a77d90-c3bd-4d1f-bd89-9c7de5ff875a None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:37:01.944388 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a77d90-c3bd-4d1f-bd89-9c7de5ff875a None admin] PATH_INFO: `/v3/projects/797f83f14e0642368d4d8f1fbd524b1c` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:37:01.945505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a77d90-c3bd-4d1f-bd89-9c7de5ff875a None admin] RBAC: Authorizing `identity:get_project(project_id=797f83f14e0642368d4d8f1fbd524b1c)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:37:01.946903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a77d90-c3bd-4d1f-bd89-9c7de5ff875a None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:37:01.948489 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 590/1191] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:37:01 2022] GET /identity/v3/projects/797f83f14e0642368d4d8f1fbd524b1c => generated 361 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:37:01.955030 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dafc0aa-fed2-443b-a3fe-e25da807e020 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:37:02.000285 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:37:02.051460 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dafc0aa-fed2-443b-a3fe-e25da807e020 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:37:02.084365 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76384a5b-bc6e-414b-8aa5-6ce74e2c3466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:37:02.085044 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76384a5b-bc6e-414b-8aa5-6ce74e2c3466 None admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:37:02.085327 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76384a5b-bc6e-414b-8aa5-6ce74e2c3466 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:37:02.085605 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76384a5b-bc6e-414b-8aa5-6ce74e2c3466 None admin] PATH_INFO: `/v3/projects/797f83f14e0642368d4d8f1fbd524b1c/users/2c89a3c49dcc40ceb3daf5d08853d995/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:37:02.087483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76384a5b-bc6e-414b-8aa5-6ce74e2c3466 None admin] RBAC: Authorizing `identity:create_grant(project_id=797f83f14e0642368d4d8f1fbd524b1c, user_id=2c89a3c49dcc40ceb3daf5d08853d995, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:37:02.088951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76384a5b-bc6e-414b-8aa5-6ce74e2c3466 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:37:02.155689 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 602/1192] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Aug 26 00:37:01 2022] PUT /identity/v3/projects/797f83f14e0642368d4d8f1fbd524b1c/users/2c89a3c49dcc40ceb3daf5d08853d995/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:37:04.020773 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1995d6-28f4-4bf2-8847-5e25fadf36c7 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:37:04.021309 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1995d6-28f4-4bf2-8847-5e25fadf36c7 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:37:04.021309 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1995d6-28f4-4bf2-8847-5e25fadf36c7 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:37:04.024795 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 591/1193] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:37:04.027903 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcd1fa-d63c-4da0-945a-f00a170f53ef None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:37:04.028286 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcd1fa-d63c-4da0-945a-f00a170f53ef None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:37:04.028451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcd1fa-d63c-4da0-945a-f00a170f53ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:37:04.046918 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-0fdcd1fa-d63c-4da0-945a-f00a170f53ef None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:37:04.048141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-0fdcd1fa-d63c-4da0-945a-f00a170f53ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:37:04.114034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 603/1194] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:37:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:37:04.380689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1995d6-28f4-4bf2-8847-5e25fadf36c7 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:37:04.493907 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1995d6-28f4-4bf2-8847-5e25fadf36c7 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:37:04.508065 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:37:04.526764 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8487bd-63ef-49a3-b7ed-a17483acf625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:37:04.527391 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8487bd-63ef-49a3-b7ed-a17483acf625 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:37:04.527673 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8487bd-63ef-49a3-b7ed-a17483acf625 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:37:04.527945 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8487bd-63ef-49a3-b7ed-a17483acf625 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:37:04.538239 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8487bd-63ef-49a3-b7ed-a17483acf625 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:37:04.542872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-5e8487bd-63ef-49a3-b7ed-a17483acf625 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:37:04.542872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:37:04.542872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:37:04.542872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:37:04.542872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:37:04.542872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:37:04.542872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:37:04.542872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:37:04.542872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:37:04.542872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:37:04.542872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:37:04.542872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:37:04.542872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:37:04.542872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:37:04.542872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:37:04.542872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:37:04.544581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:37:04.544581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:37:04.544581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:37:04.544581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:37:04.544581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:37:04.544581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:37:04.544581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:37:04.544581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:37:04.544581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:37:04.544581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:37:04.544581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:37:04.544581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:37:04.544581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:37:04.544581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:37:04.544581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:37:04.544581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:37:04.545709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:37:04.545709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:37:04.545709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:37:04.545709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:37:04.545709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:37:04.545709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:37:04.545709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:37:04.545709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:37:04.545709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:37:04.545709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:37:04.545709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:37:04.545709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:37:04.545709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 592/1195] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:37:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:37:04.550662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdcd1fa-d63c-4da0-945a-f00a170f53ef None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:37:04.584783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:37:04.648406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdcd1fa-d63c-4da0-945a-f00a170f53ef None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:37:04.681264 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b797d1e-0582-4213-8843-42314384f1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:37:04.681854 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b797d1e-0582-4213-8843-42314384f1b1 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:37:04.682129 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b797d1e-0582-4213-8843-42314384f1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:37:04.682396 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b797d1e-0582-4213-8843-42314384f1b1 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:37:04.683047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b797d1e-0582-4213-8843-42314384f1b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:37:04.683343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b797d1e-0582-4213-8843-42314384f1b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:37:04.685295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b797d1e-0582-4213-8843-42314384f1b1 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:37:04.692954 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 604/1196] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:37:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:37:04.698740 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8487bd-63ef-49a3-b7ed-a17483acf625 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:37:04.795144 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8487bd-63ef-49a3-b7ed-a17483acf625 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:37:04.827814 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7bf850-4395-4ab7-9f59-ab12a81ed904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:37:04.828428 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7bf850-4395-4ab7-9f59-ab12a81ed904 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:37:04.828738 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7bf850-4395-4ab7-9f59-ab12a81ed904 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:37:04.829012 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7bf850-4395-4ab7-9f59-ab12a81ed904 None admin] PATH_INFO: `/v3/users/2c89a3c49dcc40ceb3daf5d08853d995` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:37:04.830127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7bf850-4395-4ab7-9f59-ab12a81ed904 None admin] RBAC: Authorizing `identity:get_user(user_id=2c89a3c49dcc40ceb3daf5d08853d995)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:37:04.831550 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7bf850-4395-4ab7-9f59-ab12a81ed904 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:37:04.833149 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 593/1197] 10.222.0.59 () {56 vars in 1177 bytes} [Fri Aug 26 00:37:04 2022] GET /identity/v3/users/2c89a3c49dcc40ceb3daf5d08853d995 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:37:04.839095 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b797d1e-0582-4213-8843-42314384f1b1 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:37:04.934985 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:37:04.938703 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b797d1e-0582-4213-8843-42314384f1b1 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:37:04.972225 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11361c45-e26d-483a-8abd-fd623fe2532d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:37:04.972859 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11361c45-e26d-483a-8abd-fd623fe2532d None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:37:04.973143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11361c45-e26d-483a-8abd-fd623fe2532d None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:37:04.973417 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11361c45-e26d-483a-8abd-fd623fe2532d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:37:05.038020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11361c45-e26d-483a-8abd-fd623fe2532d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:37:05.039465 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11361c45-e26d-483a-8abd-fd623fe2532d None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:37:05.129996 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:37:05.134137 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 605/1198] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:37:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 00:37:05.140090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7bf850-4395-4ab7-9f59-ab12a81ed904 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:37:05.183312 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:37:05.238323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7bf850-4395-4ab7-9f59-ab12a81ed904 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:37:05.269987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eeccb2-002d-41e1-95fb-84b90d27178f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:37:05.270582 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eeccb2-002d-41e1-95fb-84b90d27178f None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:37:05.270859 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eeccb2-002d-41e1-95fb-84b90d27178f None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:37:05.271125 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eeccb2-002d-41e1-95fb-84b90d27178f None admin] PATH_INFO: `/v3/projects/797f83f14e0642368d4d8f1fbd524b1c` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:37:05.272201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eeccb2-002d-41e1-95fb-84b90d27178f None admin] RBAC: Authorizing `identity:get_project(project_id=797f83f14e0642368d4d8f1fbd524b1c)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:37:05.273706 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eeccb2-002d-41e1-95fb-84b90d27178f None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:37:05.275281 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 594/1199] 10.222.0.59 () {56 vars in 1186 bytes} [Fri Aug 26 00:37:05 2022] GET /identity/v3/projects/797f83f14e0642368d4d8f1fbd524b1c => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:37:05.282196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11361c45-e26d-483a-8abd-fd623fe2532d None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:37:05.377590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11361c45-e26d-483a-8abd-fd623fe2532d None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:37:05.412105 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab4672-76af-4d20-970d-e76cc13d4440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:37:05.412722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab4672-76af-4d20-970d-e76cc13d4440 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:37:05.413000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab4672-76af-4d20-970d-e76cc13d4440 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:37:05.413269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab4672-76af-4d20-970d-e76cc13d4440 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:37:05.413986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab4672-76af-4d20-970d-e76cc13d4440 None admin] RBAC: Adding query filter params (role.id=1a33e3c8cb8347feb9498595886cc621, scope.project.id=797f83f14e0642368d4d8f1fbd524b1c, user.id=2c89a3c49dcc40ceb3daf5d08853d995) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:37:05.414291 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab4672-76af-4d20-970d-e76cc13d4440 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:37:05.415697 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab4672-76af-4d20-970d-e76cc13d4440 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:37:05.421833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 606/1200] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:37:05 2022] GET /identity/v3/role_assignments?user.id=2c89a3c49dcc40ceb3daf5d08853d995&scope.project.id=797f83f14e0642368d4d8f1fbd524b1c&role.id=1a33e3c8cb8347feb9498595886cc621 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:37:15.266003 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=45 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:37:15.419035 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=69 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:18.316366 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93934b7b-d93e-44fd-8b9e-402c9c6632f2 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:18.316984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93934b7b-d93e-44fd-8b9e-402c9c6632f2 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:18.316984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93934b7b-d93e-44fd-8b9e-402c9c6632f2 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:18.318353 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 595/1201] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:38:18.323655 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c214e6e-2ac9-411e-9471-3a740785f31a None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:18.324626 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c214e6e-2ac9-411e-9471-3a740785f31a None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:18.324840 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c214e6e-2ac9-411e-9471-3a740785f31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:18.343231 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-3c214e6e-2ac9-411e-9471-3a740785f31a None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:38:18.344402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-3c214e6e-2ac9-411e-9471-3a740785f31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:38:18.410016 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 607/1202] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:38:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 00:38:18.677366 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93934b7b-d93e-44fd-8b9e-402c9c6632f2 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:18.782025 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93934b7b-d93e-44fd-8b9e-402c9c6632f2 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:18.816434 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1ff7f-ac40-47a3-ae4c-6687c3c815f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:18.817091 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1ff7f-ac40-47a3-ae4c-6687c3c815f4 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:18.817372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1ff7f-ac40-47a3-ae4c-6687c3c815f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:18.817647 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1ff7f-ac40-47a3-ae4c-6687c3c815f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:18.827526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb1ff7f-ac40-47a3-ae4c-6687c3c815f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:18.828990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb1ff7f-ac40-47a3-ae4c-6687c3c815f4 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:18.843234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-3fb1ff7f-ac40-47a3-ae4c-6687c3c815f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:38:18.843234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:18.843234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:18.843234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:18.843234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:18.843234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:18.843234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:18.843234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:18.843234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:18.843234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:18.843234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:18.843234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:18.843234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:38:18.843234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:38:18.843234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:38:18.843234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:38:18.857583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:18.857583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:18.857583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:18.857583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:18.857583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:18.857583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:18.857583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:18.857583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:18.857583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:18.857583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:18.857583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:18.857583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:18.857583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:18.857583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:18.857583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:18.857583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:18.857583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:38:18.858968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:38:18.858968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:38:18.858968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:38:18.858968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 596/1203] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:38:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:18.859278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c214e6e-2ac9-411e-9471-3a740785f31a None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:18.950607 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c214e6e-2ac9-411e-9471-3a740785f31a None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:18.979202 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:18.985332 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d6519-f245-4ba8-bdc6-01198452d567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:18.985946 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d6519-f245-4ba8-bdc6-01198452d567 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:18.986223 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d6519-f245-4ba8-bdc6-01198452d567 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:18.986495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d6519-f245-4ba8-bdc6-01198452d567 None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:18.987131 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d6519-f245-4ba8-bdc6-01198452d567 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:18.987432 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d6519-f245-4ba8-bdc6-01198452d567 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:18.988828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d6519-f245-4ba8-bdc6-01198452d567 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:19.000275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 608/1204] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:19.006910 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1ff7f-ac40-47a3-ae4c-6687c3c815f4 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:19.060834 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:19.108208 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1ff7f-ac40-47a3-ae4c-6687c3c815f4 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:19.141150 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f5499-3f38-42ea-9667-9bb66ec24e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:19.141754 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f5499-3f38-42ea-9667-9bb66ec24e57 None admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:19.142035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f5499-3f38-42ea-9667-9bb66ec24e57 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:19.142309 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f5499-3f38-42ea-9667-9bb66ec24e57 None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:19.142942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417f5499-3f38-42ea-9667-9bb66ec24e57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:19.144403 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417f5499-3f38-42ea-9667-9bb66ec24e57 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:19.288493 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 597/1205] 10.222.0.59 () {60 vars in 1132 bytes} [Fri Aug 26 00:38:19 2022] POST /identity/v3/users => generated 251 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:38:21.239304 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e1cdd-8215-4192-9cad-3ad382a4a3c6 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:21.239860 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e1cdd-8215-4192-9cad-3ad382a4a3c6 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:21.239860 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e1cdd-8215-4192-9cad-3ad382a4a3c6 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:21.241308 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 609/1206] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:38:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:38:21.246651 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f6c8a-3da3-4bfe-b3a2-646aaabbafda None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:21.247070 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f6c8a-3da3-4bfe-b3a2-646aaabbafda None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:21.247180 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f6c8a-3da3-4bfe-b3a2-646aaabbafda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:21.265852 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-224f6c8a-3da3-4bfe-b3a2-646aaabbafda None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:38:21.267039 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-224f6c8a-3da3-4bfe-b3a2-646aaabbafda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:38:21.334048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 598/1207] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:38:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:38:21.605144 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e1cdd-8215-4192-9cad-3ad382a4a3c6 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:21.705500 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e1cdd-8215-4192-9cad-3ad382a4a3c6 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:21.738326 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d0b7f-ff0a-4637-a6a0-76eb7f6dbd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:21.738951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d0b7f-ff0a-4637-a6a0-76eb7f6dbd76 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:21.739225 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d0b7f-ff0a-4637-a6a0-76eb7f6dbd76 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:21.739490 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d0b7f-ff0a-4637-a6a0-76eb7f6dbd76 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:21.750671 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5d0b7f-ff0a-4637-a6a0-76eb7f6dbd76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:38:21.755356 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-aa5d0b7f-ff0a-4637-a6a0-76eb7f6dbd76 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:38:21.755356 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:21.755356 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:21.755356 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:21.755356 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:21.755356 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:21.755356 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:21.755356 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:21.755356 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:21.755356 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:21.755356 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:21.755356 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:21.755356 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:38:21.755356 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:38:21.755356 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:38:21.755356 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:38:21.756559 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:38:21.756559 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:38:21.756559 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:38:21.756559 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:38:21.756559 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:21.756559 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:21.756559 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:21.756559 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:21.756559 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:21.756559 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:21.756559 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:21.756559 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:21.756559 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:21.756559 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:21.756559 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:21.756559 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:21.757821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:21.757821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:21.757821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:21.757821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:21.757821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:38:21.757821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:38:21.757821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:38:21.757821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:38:21.757821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:38:21.757821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:38:21.757821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:38:21.757821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:38:21.757821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 610/1208] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Aug 26 00:38:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:21.763018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f6c8a-3da3-4bfe-b3a2-646aaabbafda None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:21.801437 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:21.860781 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f6c8a-3da3-4bfe-b3a2-646aaabbafda None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:21.894369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61d017-287a-4acb-bf37-cbc6a473b2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:21.894969 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61d017-287a-4acb-bf37-cbc6a473b2bd None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:21.895248 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61d017-287a-4acb-bf37-cbc6a473b2bd None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:21.895522 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61d017-287a-4acb-bf37-cbc6a473b2bd None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:21.896175 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61d017-287a-4acb-bf37-cbc6a473b2bd None admin] RBAC: Adding query filter params (name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:21.896534 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61d017-287a-4acb-bf37-cbc6a473b2bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:21.897900 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61d017-287a-4acb-bf37-cbc6a473b2bd None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:21.906381 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 599/1209] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Aug 26 00:38:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:21.912104 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d0b7f-ff0a-4637-a6a0-76eb7f6dbd76 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:21.998384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:22.046583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d0b7f-ff0a-4637-a6a0-76eb7f6dbd76 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:22.078858 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1160cc7-60b3-4fcf-8dc0-2709687905d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:22.079484 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1160cc7-60b3-4fcf-8dc0-2709687905d0 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:22.079767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1160cc7-60b3-4fcf-8dc0-2709687905d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:22.080042 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1160cc7-60b3-4fcf-8dc0-2709687905d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:22.090859 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1160cc7-60b3-4fcf-8dc0-2709687905d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:22.092325 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1160cc7-60b3-4fcf-8dc0-2709687905d0 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:22.105889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-a1160cc7-60b3-4fcf-8dc0-2709687905d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:38:22.105889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:22.105889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:22.105889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:22.105889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:22.105889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:22.105889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:22.105889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:22.105889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:22.105889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:22.105889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:22.105889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:22.105889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:38:22.105889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:38:22.105889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:38:22.105889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:38:22.111870 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:22.111870 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:22.111870 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:22.111870 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:22.111870 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:22.111870 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:22.111870 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:22.111870 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:22.111870 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:22.111870 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:22.111870 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:22.111870 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:22.111870 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:22.111870 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:22.111870 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:22.111870 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:22.111870 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:38:22.113101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:38:22.113101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:38:22.113101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:38:22.113101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 611/1210] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:38:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:22.113397 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61d017-287a-4acb-bf37-cbc6a473b2bd None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:22.209181 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:22.212021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61d017-287a-4acb-bf37-cbc6a473b2bd None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:22.248357 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78bd8fc-7ca2-452e-82a8-582c083fda18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:22.249000 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78bd8fc-7ca2-452e-82a8-582c083fda18 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:22.249259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78bd8fc-7ca2-452e-82a8-582c083fda18 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:22.249530 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78bd8fc-7ca2-452e-82a8-582c083fda18 None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:22.250192 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78bd8fc-7ca2-452e-82a8-582c083fda18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:22.250490 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78bd8fc-7ca2-452e-82a8-582c083fda18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:22.251838 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78bd8fc-7ca2-452e-82a8-582c083fda18 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:22.264694 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 600/1211] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:38:22.270372 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1160cc7-60b3-4fcf-8dc0-2709687905d0 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:22.374096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1160cc7-60b3-4fcf-8dc0-2709687905d0 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:22.375551 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:22.412743 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c0dce-1867-49fa-8f8c-5cf7aa49a494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:22.413376 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c0dce-1867-49fa-8f8c-5cf7aa49a494 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:22.413657 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c0dce-1867-49fa-8f8c-5cf7aa49a494 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:22.413932 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c0dce-1867-49fa-8f8c-5cf7aa49a494 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:22.420948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c0dce-1867-49fa-8f8c-5cf7aa49a494 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:22.422382 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c0dce-1867-49fa-8f8c-5cf7aa49a494 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:22.434076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-e72c0dce-1867-49fa-8f8c-5cf7aa49a494 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 00:38:22.434076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:22.434076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:22.434076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:22.434076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:22.434076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:22.434076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:22.434076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:22.434076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:22.434076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:22.434076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:22.434076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:22.434076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:38:22.434076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:38:22.434076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:38:22.434076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:38:22.435114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:22.435114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:22.435114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:38:22.435114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:38:22.435114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:38:22.435114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:38:22.435114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:22.435114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:22.435114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:22.435114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:22.435114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:22.435114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:22.435114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:22.435114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:22.435114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:22.435114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:22.435114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:22.435114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:22.436566 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:22.436566 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:22.436566 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:38:22.436566 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:38:22.436566 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:38:22.436566 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:38:22.436566 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:38:22.436566 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:38:22.436566 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 00:38:22.436566 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:38:22.436566 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 612/1212] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:38:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:22.441124 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78bd8fc-7ca2-452e-82a8-582c083fda18 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:22.537457 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78bd8fc-7ca2-452e-82a8-582c083fda18 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:22.571037 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518164c-4ce0-4565-8a8c-2af79da1092e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:22.571637 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518164c-4ce0-4565-8a8c-2af79da1092e None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:22.571913 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518164c-4ce0-4565-8a8c-2af79da1092e None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:22.572182 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518164c-4ce0-4565-8a8c-2af79da1092e None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:22.572931 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1518164c-4ce0-4565-8a8c-2af79da1092e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:22.573236 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1518164c-4ce0-4565-8a8c-2af79da1092e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:22.574639 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1518164c-4ce0-4565-8a8c-2af79da1092e None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:22.591444 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 601/1213] 10.222.0.59 () {56 vars in 1143 bytes} [Fri Aug 26 00:38:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:22.597242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c0dce-1867-49fa-8f8c-5cf7aa49a494 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:22.697651 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c0dce-1867-49fa-8f8c-5cf7aa49a494 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:22.730620 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db464a3-1d1a-4bd5-8107-f17454305d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:22.731369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db464a3-1d1a-4bd5-8107-f17454305d44 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:22.731668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db464a3-1d1a-4bd5-8107-f17454305d44 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:22.731930 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db464a3-1d1a-4bd5-8107-f17454305d44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:22.761803 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:22.799383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db464a3-1d1a-4bd5-8107-f17454305d44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:22.800946 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db464a3-1d1a-4bd5-8107-f17454305d44 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:22.900110 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 613/1214] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:38:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 00:38:22.906335 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518164c-4ce0-4565-8a8c-2af79da1092e None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:22.924871 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:23.005210 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518164c-4ce0-4565-8a8c-2af79da1092e None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:23.038790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa761e-729c-4a2e-9aab-09dc070b458c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:23.039411 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa761e-729c-4a2e-9aab-09dc070b458c None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:23.039689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa761e-729c-4a2e-9aab-09dc070b458c None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:23.039960 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa761e-729c-4a2e-9aab-09dc070b458c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:23.049719 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa761e-729c-4a2e-9aab-09dc070b458c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:23.051750 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa761e-729c-4a2e-9aab-09dc070b458c None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:23.066090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-adfa761e-729c-4a2e-9aab-09dc070b458c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:38:23.066090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:23.066090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:23.066090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:23.066090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:23.066090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:23.066090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:23.066090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:23.066090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:23.066090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:23.066090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:23.066090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:23.066090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:38:23.066090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:38:23.066090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:38:23.066090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:38:23.067315 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:23.067315 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:23.067315 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:23.067315 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:23.067315 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:23.067315 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:23.067315 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:23.067315 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:23.067315 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:23.067315 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:23.067315 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:23.067315 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:23.067315 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:23.067315 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:23.067315 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:23.067315 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:23.067315 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:38:23.068267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:38:23.068267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:38:23.068267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:38:23.068267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 602/1215] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:38:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:23.073203 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db464a3-1d1a-4bd5-8107-f17454305d44 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:23.133996 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:23.171748 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db464a3-1d1a-4bd5-8107-f17454305d44 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:23.204535 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca6dff-4e71-4b7d-bd11-7629b7b2ce4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:23.205147 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca6dff-4e71-4b7d-bd11-7629b7b2ce4f None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:23.205431 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca6dff-4e71-4b7d-bd11-7629b7b2ce4f None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:23.205706 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca6dff-4e71-4b7d-bd11-7629b7b2ce4f None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:23.206350 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca6dff-4e71-4b7d-bd11-7629b7b2ce4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:23.206660 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca6dff-4e71-4b7d-bd11-7629b7b2ce4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:23.208021 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca6dff-4e71-4b7d-bd11-7629b7b2ce4f None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:23.219787 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 614/1216] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:23.225598 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa761e-729c-4a2e-9aab-09dc070b458c None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:23.293675 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:23.323768 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa761e-729c-4a2e-9aab-09dc070b458c None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:23.356354 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fed250-b026-45c7-9985-99c653eb5420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:23.357012 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fed250-b026-45c7-9985-99c653eb5420 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:23.357293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fed250-b026-45c7-9985-99c653eb5420 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:23.357567 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fed250-b026-45c7-9985-99c653eb5420 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:23.364642 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fed250-b026-45c7-9985-99c653eb5420 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:23.366123 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fed250-b026-45c7-9985-99c653eb5420 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:23.377654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-d6fed250-b026-45c7-9985-99c653eb5420 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:38:23.377654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:23.377654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:23.377654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:23.377654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:23.377654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:23.377654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:23.377654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:23.377654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:23.377654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:23.377654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:23.377654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:23.377654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:38:23.377654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:38:23.377654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:38:23.382358 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:38:23.382358 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:23.382358 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:23.382358 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:23.382358 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:23.382358 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:23.382358 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:23.382358 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:23.382358 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:23.382358 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:23.382358 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:23.382358 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:23.382358 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:23.382358 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:23.382358 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:23.382358 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:23.382358 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:23.383683 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:38:23.383683 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:38:23.383683 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:38:23.383683 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:38:23.383683 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:38:23.383683 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:38:23.383683 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:38:23.383683 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:38:23.383683 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 603/1217] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:38:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:23.384968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca6dff-4e71-4b7d-bd11-7629b7b2ce4f None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:23.485497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca6dff-4e71-4b7d-bd11-7629b7b2ce4f None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:23.507725 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:23.518782 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd21bc-b85e-4964-ae86-fafd983cab3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:23.519383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd21bc-b85e-4964-ae86-fafd983cab3c None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:23.519660 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd21bc-b85e-4964-ae86-fafd983cab3c None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:23.519929 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd21bc-b85e-4964-ae86-fafd983cab3c None admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:23.520763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd21bc-b85e-4964-ae86-fafd983cab3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:23.521068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd21bc-b85e-4964-ae86-fafd983cab3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:23.522488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd21bc-b85e-4964-ae86-fafd983cab3c None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:23.533740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 615/1218] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:38:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:23.539997 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fed250-b026-45c7-9985-99c653eb5420 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:23.646346 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fed250-b026-45c7-9985-99c653eb5420 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:23.675286 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:23.680929 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05b686-2d71-43e4-bfb2-635155b9e2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:23.681531 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05b686-2d71-43e4-bfb2-635155b9e2df None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:23.681810 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05b686-2d71-43e4-bfb2-635155b9e2df None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:23.682086 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05b686-2d71-43e4-bfb2-635155b9e2df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:23.682798 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab05b686-2d71-43e4-bfb2-635155b9e2df None admin] RBAC: Adding query filter params (role.id=bce78c7e0a534135a820de7f3588aa4d, scope.project.id=fde7ac3ed472433ab3bba56e5e091510, user.id=2876f4182f0949039b862f053533fe85) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:23.683103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab05b686-2d71-43e4-bfb2-635155b9e2df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:23.684577 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab05b686-2d71-43e4-bfb2-635155b9e2df None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:23.689692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 604/1219] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:38:23 2022] GET /identity/v3/role_assignments?user.id=2876f4182f0949039b862f053533fe85&scope.project.id=fde7ac3ed472433ab3bba56e5e091510&role.id=bce78c7e0a534135a820de7f3588aa4d => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:25.574638 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6839266-1d4b-4d82-b8fd-8db1e94d1be4 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:25.575266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6839266-1d4b-4d82-b8fd-8db1e94d1be4 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:25.575266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6839266-1d4b-4d82-b8fd-8db1e94d1be4 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:25.576765 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 616/1220] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:38:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:38:25.582018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf6a9a-ceef-44fe-b778-b10bcab2b17a None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:25.582483 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf6a9a-ceef-44fe-b778-b10bcab2b17a None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:25.582669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf6a9a-ceef-44fe-b778-b10bcab2b17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:25.600936 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-04cf6a9a-ceef-44fe-b778-b10bcab2b17a None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:38:25.602117 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-04cf6a9a-ceef-44fe-b778-b10bcab2b17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:38:25.666939 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 605/1221] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:38:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:38:25.944144 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6839266-1d4b-4d82-b8fd-8db1e94d1be4 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:26.043522 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6839266-1d4b-4d82-b8fd-8db1e94d1be4 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:26.077748 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdab03a-5603-4a8a-a5ce-204f9ed33732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:26.078374 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdab03a-5603-4a8a-a5ce-204f9ed33732 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:26.078651 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdab03a-5603-4a8a-a5ce-204f9ed33732 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:26.078920 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdab03a-5603-4a8a-a5ce-204f9ed33732 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:26.088656 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdab03a-5603-4a8a-a5ce-204f9ed33732 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:38:26.093812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-abdab03a-5603-4a8a-a5ce-204f9ed33732 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:38:26.093812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:26.093812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:26.093812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:26.093812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:26.093812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:26.093812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:26.093812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:26.093812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:26.093812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:26.093812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:26.093812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:26.093812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:38:26.093812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:38:26.093812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:38:26.093812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:38:26.095080 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:38:26.095080 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:38:26.095080 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:38:26.095080 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:38:26.095080 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:26.095080 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:26.095080 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:26.095080 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:26.095080 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:26.095080 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:26.095080 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:26.095080 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:26.095080 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:26.095080 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:26.095080 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:26.095080 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:26.096169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:26.096169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:26.096169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:26.096169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:26.096169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:38:26.096169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:38:26.096169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:38:26.096169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:38:26.096169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:38:26.096169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:38:26.096169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:38:26.096169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:38:26.096169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 617/1222] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Aug 26 00:38:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:26.101349 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf6a9a-ceef-44fe-b778-b10bcab2b17a None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:26.197512 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf6a9a-ceef-44fe-b778-b10bcab2b17a None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:26.221611 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:26.231710 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db936ec2-f03b-4692-a9e9-96a7ee5559ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:26.232303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db936ec2-f03b-4692-a9e9-96a7ee5559ff None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:26.232606 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db936ec2-f03b-4692-a9e9-96a7ee5559ff None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:26.232876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db936ec2-f03b-4692-a9e9-96a7ee5559ff None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:26.233533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db936ec2-f03b-4692-a9e9-96a7ee5559ff None admin] RBAC: Adding query filter params (name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:26.233833 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db936ec2-f03b-4692-a9e9-96a7ee5559ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:26.235250 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db936ec2-f03b-4692-a9e9-96a7ee5559ff None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:26.246041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 606/1223] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Aug 26 00:38:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:26.248455 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdab03a-5603-4a8a-a5ce-204f9ed33732 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:26.292951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:26.347125 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdab03a-5603-4a8a-a5ce-204f9ed33732 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:26.380672 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d3fb9-ce39-4d57-827c-68de0cc90483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:26.381348 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d3fb9-ce39-4d57-827c-68de0cc90483 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:26.381596 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d3fb9-ce39-4d57-827c-68de0cc90483 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:26.381855 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d3fb9-ce39-4d57-827c-68de0cc90483 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:26.392153 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d3fb9-ce39-4d57-827c-68de0cc90483 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:26.399742 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d3fb9-ce39-4d57-827c-68de0cc90483 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:26.408700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-c86d3fb9-ce39-4d57-827c-68de0cc90483 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:38:26.408700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:26.408700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:26.408700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:26.408700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:26.408700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:26.408700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:26.408700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:26.408700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:26.408700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:26.408700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:26.408700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:26.408700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:38:26.408700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:38:26.408700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:38:26.408700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:38:26.409910 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:26.409910 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:26.409910 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:26.409910 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:26.409910 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:26.409910 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:26.409910 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:26.409910 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:26.409910 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:26.409910 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:26.409910 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:26.409910 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:26.409910 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:26.409910 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:26.409910 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:26.409910 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:26.409910 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:38:26.411015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:38:26.411015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:38:26.411015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:38:26.411015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 618/1224] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:38:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:26.415560 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db936ec2-f03b-4692-a9e9-96a7ee5559ff None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:26.514870 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db936ec2-f03b-4692-a9e9-96a7ee5559ff None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:26.547796 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb64e4b-efcd-4253-8bd0-3691299122cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:26.548399 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb64e4b-efcd-4253-8bd0-3691299122cb None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:26.548714 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb64e4b-efcd-4253-8bd0-3691299122cb None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:26.548998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb64e4b-efcd-4253-8bd0-3691299122cb None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:26.549686 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb64e4b-efcd-4253-8bd0-3691299122cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:26.549991 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb64e4b-efcd-4253-8bd0-3691299122cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:26.551346 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb64e4b-efcd-4253-8bd0-3691299122cb None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:26.563488 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 607/1225] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:26.569245 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d3fb9-ce39-4d57-827c-68de0cc90483 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:26.660778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:26.667227 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d3fb9-ce39-4d57-827c-68de0cc90483 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:26.699560 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb382829-8062-4867-a4b5-8720f9c4b0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:26.700182 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb382829-8062-4867-a4b5-8720f9c4b0a0 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:26.700462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb382829-8062-4867-a4b5-8720f9c4b0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:26.700760 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb382829-8062-4867-a4b5-8720f9c4b0a0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:26.707431 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb382829-8062-4867-a4b5-8720f9c4b0a0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:26.708900 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb382829-8062-4867-a4b5-8720f9c4b0a0 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:26.720430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-fb382829-8062-4867-a4b5-8720f9c4b0a0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 00:38:26.720430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:26.720430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:26.720430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:26.720430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:26.720430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:26.720430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:26.720430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:26.720430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:26.720430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:26.720430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:26.720430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:26.720430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:38:26.720430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:38:26.720430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:38:26.720430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:38:26.726473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:26.726473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:26.726473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:38:26.726473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:38:26.726473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:38:26.726473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:38:26.726473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:26.726473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:26.726473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:26.726473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:26.726473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:26.726473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:26.726473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:26.726473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:26.726473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:26.726473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:26.726473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:26.726473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:26.727730 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:26.727730 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:26.727730 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:38:26.727730 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:38:26.727730 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:38:26.727730 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:38:26.727730 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:38:26.727730 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:38:26.727730 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 00:38:26.727730 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:38:26.727730 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 619/1226] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:38:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:26.728652 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb64e4b-efcd-4253-8bd0-3691299122cb None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:26.772759 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:26.828489 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb64e4b-efcd-4253-8bd0-3691299122cb None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:26.862490 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3375e-a867-4682-a082-5ef460563662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:26.863088 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3375e-a867-4682-a082-5ef460563662 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:26.863369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3375e-a867-4682-a082-5ef460563662 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:26.863650 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3375e-a867-4682-a082-5ef460563662 None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:26.864369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e3375e-a867-4682-a082-5ef460563662 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:26.864702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e3375e-a867-4682-a082-5ef460563662 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:26.866125 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e3375e-a867-4682-a082-5ef460563662 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:26.882774 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 608/1227] 10.222.0.59 () {56 vars in 1143 bytes} [Fri Aug 26 00:38:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:26.888514 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb382829-8062-4867-a4b5-8720f9c4b0a0 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:26.987267 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb382829-8062-4867-a4b5-8720f9c4b0a0 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:27.022907 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d78ff4-476a-43de-9553-c6139e32cf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:27.023467 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d78ff4-476a-43de-9553-c6139e32cf47 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:27.023745 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d78ff4-476a-43de-9553-c6139e32cf47 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:27.024018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d78ff4-476a-43de-9553-c6139e32cf47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:27.035102 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d78ff4-476a-43de-9553-c6139e32cf47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:27.036595 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d78ff4-476a-43de-9553-c6139e32cf47 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:27.052576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-26d78ff4-476a-43de-9553-c6139e32cf47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:38:27.052576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:27.052576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:27.052576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:27.052576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:27.052576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:27.052576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:27.052576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:27.052576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:27.052576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:27.052576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:27.052576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:27.052576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:38:27.052576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:38:27.052576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:38:27.052576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:38:27.053795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:27.053795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:27.053795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:27.053795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:27.053795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:27.053795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:27.053795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:27.053795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:27.053795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:27.053795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:27.053795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:27.053795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:27.053795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:27.053795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:27.053795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:27.053795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:27.053795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:38:27.055132 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:38:27.055132 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:38:27.055132 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:38:27.055132 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 620/1228] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:38:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:27.059650 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3375e-a867-4682-a082-5ef460563662 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:27.157630 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3375e-a867-4682-a082-5ef460563662 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:27.159968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:27.190845 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e6119-1a1e-4882-b15e-693dbec37c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:27.191444 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e6119-1a1e-4882-b15e-693dbec37c66 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:27.191723 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e6119-1a1e-4882-b15e-693dbec37c66 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:27.191993 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e6119-1a1e-4882-b15e-693dbec37c66 None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:27.192661 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e6119-1a1e-4882-b15e-693dbec37c66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:27.192962 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e6119-1a1e-4882-b15e-693dbec37c66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:27.194311 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e6119-1a1e-4882-b15e-693dbec37c66 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:27.205640 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 609/1229] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:27.211350 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d78ff4-476a-43de-9553-c6139e32cf47 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:27.222269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:27.310056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d78ff4-476a-43de-9553-c6139e32cf47 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:27.343107 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5137cb-ea00-465d-8b61-3281279c07f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:27.343729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5137cb-ea00-465d-8b61-3281279c07f2 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:27.344010 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5137cb-ea00-465d-8b61-3281279c07f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:27.344284 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5137cb-ea00-465d-8b61-3281279c07f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:27.352467 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5137cb-ea00-465d-8b61-3281279c07f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:27.353966 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5137cb-ea00-465d-8b61-3281279c07f2 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:27.364957 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-3a5137cb-ea00-465d-8b61-3281279c07f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:38:27.364957 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:27.364957 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:27.364957 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:27.364957 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:27.364957 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:27.364957 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:27.364957 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:27.364957 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:27.364957 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:27.364957 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:27.364957 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:27.364957 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:38:27.364957 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:38:27.364957 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:38:27.366055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:38:27.366055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:27.366055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:27.366055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:27.366055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:27.366055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:27.366055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:27.366055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:27.366055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:27.366055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:27.366055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:27.366055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:27.366055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:27.366055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:27.366055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:27.366055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:27.366055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:27.367133 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:38:27.367133 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:38:27.367133 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:38:27.367133 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:38:27.367133 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:38:27.367133 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:38:27.367133 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:38:27.367133 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:38:27.367133 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 621/1230] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:38:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:27.372431 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e6119-1a1e-4882-b15e-693dbec37c66 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:27.475491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e6119-1a1e-4882-b15e-693dbec37c66 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:27.508603 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87752d32-c229-4762-b875-9bae4fcb318e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:27.509203 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87752d32-c229-4762-b875-9bae4fcb318e None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:27.509484 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87752d32-c229-4762-b875-9bae4fcb318e None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:27.509760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87752d32-c229-4762-b875-9bae4fcb318e None admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:27.510446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87752d32-c229-4762-b875-9bae4fcb318e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:27.510754 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87752d32-c229-4762-b875-9bae4fcb318e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:27.512235 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87752d32-c229-4762-b875-9bae4fcb318e None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:27.524629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 610/1231] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:38:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:27.530986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5137cb-ea00-465d-8b61-3281279c07f2 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:27.596420 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:27.630108 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5137cb-ea00-465d-8b61-3281279c07f2 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:27.662605 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1dcd1c-6fc4-48fd-8a03-2d004a720636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:27.663371 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1dcd1c-6fc4-48fd-8a03-2d004a720636 None admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:27.663651 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1dcd1c-6fc4-48fd-8a03-2d004a720636 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:27.663924 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1dcd1c-6fc4-48fd-8a03-2d004a720636 None admin] PATH_INFO: `/v3/projects/fde7ac3ed472433ab3bba56e5e091510/users/2876f4182f0949039b862f053533fe85/roles/bce78c7e0a534135a820de7f3588aa4d` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:27.677716 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1dcd1c-6fc4-48fd-8a03-2d004a720636 None admin] RBAC: Authorizing `identity:create_grant(project_id=fde7ac3ed472433ab3bba56e5e091510, user_id=2876f4182f0949039b862f053533fe85, role_id=bce78c7e0a534135a820de7f3588aa4d)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:27.679145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1dcd1c-6fc4-48fd-8a03-2d004a720636 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:27.761123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 622/1232] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Aug 26 00:38:27 2022] PUT /identity/v3/projects/fde7ac3ed472433ab3bba56e5e091510/users/2876f4182f0949039b862f053533fe85/roles/bce78c7e0a534135a820de7f3588aa4d => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:38:29.637072 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7e4a3-949b-4b27-8a83-8f834846771a None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:29.637620 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7e4a3-949b-4b27-8a83-8f834846771a None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:29.637620 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7e4a3-949b-4b27-8a83-8f834846771a None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:29.638937 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 611/1233] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:38:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:38:29.644080 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16498a1b-438f-446d-a6e6-0b8b62f4d180 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:29.646581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16498a1b-438f-446d-a6e6-0b8b62f4d180 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:29.646581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16498a1b-438f-446d-a6e6-0b8b62f4d180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:29.662236 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-16498a1b-438f-446d-a6e6-0b8b62f4d180 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:38:29.663393 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-16498a1b-438f-446d-a6e6-0b8b62f4d180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:38:29.727461 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 623/1234] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:38:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 00:38:29.997380 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7e4a3-949b-4b27-8a83-8f834846771a None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:30.021525 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:30.095990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7e4a3-949b-4b27-8a83-8f834846771a None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:30.130013 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00f48e-1b50-48b9-b8a3-65c008462980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:30.130635 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00f48e-1b50-48b9-b8a3-65c008462980 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:30.130915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00f48e-1b50-48b9-b8a3-65c008462980 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:30.131187 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00f48e-1b50-48b9-b8a3-65c008462980 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:30.141065 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f00f48e-1b50-48b9-b8a3-65c008462980 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:38:30.145987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-8f00f48e-1b50-48b9-b8a3-65c008462980 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:38:30.145987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:30.145987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:30.145987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:30.145987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:30.145987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:30.145987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:30.145987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:30.145987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:30.145987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:30.145987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:30.145987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:30.145987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:38:30.145987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:38:30.145987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:38:30.145987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:38:30.147140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:38:30.147140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:38:30.147140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:38:30.147140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:38:30.147140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:30.147140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:30.147140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:30.147140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:30.147140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:30.147140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:30.147140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:30.147140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:30.147140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:30.147140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:30.147140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:30.147140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:30.148195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:30.148195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:30.148195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:30.148195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:30.148195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:38:30.148195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:38:30.148195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:38:30.148195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:38:30.148195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:38:30.148195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:38:30.148195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 00:38:30.148195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:38:30.148195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 612/1235] 10.222.0.59 () {56 vars in 1102 bytes} [Fri Aug 26 00:38:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:38:30.153292 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16498a1b-438f-446d-a6e6-0b8b62f4d180 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:30.200016 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:30.250874 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16498a1b-438f-446d-a6e6-0b8b62f4d180 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:30.285046 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e7d70-d4a3-45f0-b67a-e7de14881a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:30.285649 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e7d70-d4a3-45f0-b67a-e7de14881a2a None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:30.285920 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e7d70-d4a3-45f0-b67a-e7de14881a2a None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:30.286189 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e7d70-d4a3-45f0-b67a-e7de14881a2a None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:30.286842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378e7d70-d4a3-45f0-b67a-e7de14881a2a None admin] RBAC: Adding query filter params (name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:30.287139 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378e7d70-d4a3-45f0-b67a-e7de14881a2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:30.288508 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378e7d70-d4a3-45f0-b67a-e7de14881a2a None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:30.296040 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 624/1236] 10.222.0.59 () {56 vars in 1103 bytes} [Fri Aug 26 00:38:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:30.301873 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00f48e-1b50-48b9-b8a3-65c008462980 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:30.383667 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:30.399338 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00f48e-1b50-48b9-b8a3-65c008462980 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:30.432790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6db177-eff9-4c96-b382-2de5fb4e24e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:30.433934 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6db177-eff9-4c96-b382-2de5fb4e24e9 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:30.434300 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6db177-eff9-4c96-b382-2de5fb4e24e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:30.434625 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6db177-eff9-4c96-b382-2de5fb4e24e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:30.444460 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6db177-eff9-4c96-b382-2de5fb4e24e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:30.445952 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6db177-eff9-4c96-b382-2de5fb4e24e9 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:30.461978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-ac6db177-eff9-4c96-b382-2de5fb4e24e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:38:30.461978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:30.461978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:30.461978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:30.461978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:30.461978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:30.461978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:30.461978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:30.461978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:30.461978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:30.461978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:30.461978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:30.461978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:38:30.461978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:38:30.461978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:38:30.461978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:38:30.467948 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:30.467948 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:30.467948 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:30.467948 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:30.467948 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:30.467948 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:30.467948 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:30.467948 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:30.467948 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:30.467948 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:30.467948 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:30.467948 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:30.467948 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:30.467948 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:30.467948 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:30.467948 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:30.467948 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:38:30.469118 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:38:30.469118 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:38:30.469118 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:38:30.469118 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 613/1237] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:30.469411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e7d70-d4a3-45f0-b67a-e7de14881a2a None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:30.568008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e7d70-d4a3-45f0-b67a-e7de14881a2a None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:30.577205 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:30.601302 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a8b84-ebc9-4baa-a523-9da61861729a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:30.601931 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a8b84-ebc9-4baa-a523-9da61861729a None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:30.602214 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a8b84-ebc9-4baa-a523-9da61861729a None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:30.602490 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a8b84-ebc9-4baa-a523-9da61861729a None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:30.603124 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a8b84-ebc9-4baa-a523-9da61861729a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:30.603424 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a8b84-ebc9-4baa-a523-9da61861729a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:30.604823 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a8b84-ebc9-4baa-a523-9da61861729a None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:30.616290 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 625/1238] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:38:30.622761 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6db177-eff9-4c96-b382-2de5fb4e24e9 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:30.724858 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6db177-eff9-4c96-b382-2de5fb4e24e9 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:30.763191 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03eb6d-2578-47cf-8f68-6d7e0719a4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:30.763813 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03eb6d-2578-47cf-8f68-6d7e0719a4fb None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:30.764096 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03eb6d-2578-47cf-8f68-6d7e0719a4fb None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:30.764371 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03eb6d-2578-47cf-8f68-6d7e0719a4fb None admin] PATH_INFO: `/v3/users/placement` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:30.767011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:30.771770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03eb6d-2578-47cf-8f68-6d7e0719a4fb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:30.773254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03eb6d-2578-47cf-8f68-6d7e0719a4fb None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:30.784250 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-2a03eb6d-2578-47cf-8f68-6d7e0719a4fb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 00:38:30.784250 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:30.784250 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:30.784250 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:30.784250 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:30.784250 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:30.784250 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:30.784250 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:30.784250 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:30.784250 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:30.784250 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:30.784250 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:30.784250 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:38:30.784250 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:38:30.784250 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:38:30.784250 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:38:30.785433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:30.785433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:30.785433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:38:30.785433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:38:30.785433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:38:30.785433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:38:30.785433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:30.785433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:30.785433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:30.785433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:30.785433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:30.785433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:30.785433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:30.785433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:30.785433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:30.785433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:30.785433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:30.785433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:30.786628 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:30.786628 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:30.786628 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:38:30.786628 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:38:30.786628 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:38:30.786628 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:38:30.786628 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:38:30.786628 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:38:30.786628 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 00:38:30.786628 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:38:30.786628 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 614/1239] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:38:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:30.791439 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a8b84-ebc9-4baa-a523-9da61861729a None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:30.890660 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a8b84-ebc9-4baa-a523-9da61861729a None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:30.924328 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88812e-51e2-4ae9-8634-30457e22e4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:30.924957 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88812e-51e2-4ae9-8634-30457e22e4a9 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:30.925239 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88812e-51e2-4ae9-8634-30457e22e4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:30.925512 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88812e-51e2-4ae9-8634-30457e22e4a9 None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:30.926234 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88812e-51e2-4ae9-8634-30457e22e4a9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:30.926536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88812e-51e2-4ae9-8634-30457e22e4a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:30.927954 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88812e-51e2-4ae9-8634-30457e22e4a9 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:30.945061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 626/1240] 10.222.0.59 () {56 vars in 1143 bytes} [Fri Aug 26 00:38:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:30.950903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03eb6d-2578-47cf-8f68-6d7e0719a4fb None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:31.049460 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03eb6d-2578-47cf-8f68-6d7e0719a4fb None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:31.082700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e8416d-4800-46c9-b955-c4f7f76e51c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:31.083302 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8416d-4800-46c9-b955-c4f7f76e51c6 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:31.083578 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8416d-4800-46c9-b955-c4f7f76e51c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:31.083884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8416d-4800-46c9-b955-c4f7f76e51c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:31.142838 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:31.149007 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e8416d-4800-46c9-b955-c4f7f76e51c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:31.150457 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e8416d-4800-46c9-b955-c4f7f76e51c6 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:31.246350 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 615/1241] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:38:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 00:38:31.252438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88812e-51e2-4ae9-8634-30457e22e4a9 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:31.279944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:31.350771 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88812e-51e2-4ae9-8634-30457e22e4a9 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:31.389903 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63242cf1-585e-4783-9f9c-fb3bf1b448a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:31.390552 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63242cf1-585e-4783-9f9c-fb3bf1b448a9 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:31.390830 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63242cf1-585e-4783-9f9c-fb3bf1b448a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:31.391100 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63242cf1-585e-4783-9f9c-fb3bf1b448a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:31.402352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63242cf1-585e-4783-9f9c-fb3bf1b448a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:31.403738 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63242cf1-585e-4783-9f9c-fb3bf1b448a9 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:31.419043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-63242cf1-585e-4783-9f9c-fb3bf1b448a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:38:31.419043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:31.419043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:31.419043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:31.419043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:31.419043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:31.419043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:31.419043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:31.419043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:31.419043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:31.419043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:31.419043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:31.419043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:38:31.419043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:38:31.419043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:38:31.419043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:38:31.420244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:31.420244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:31.420244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:31.420244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:31.420244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:31.420244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:31.420244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:31.420244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:31.420244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:31.420244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:31.420244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:31.420244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:31.420244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:31.420244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:31.420244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:31.420244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:31.420244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:38:31.421441 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:38:31.421441 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:38:31.421441 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:38:31.424619 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 627/1242] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:38:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:31.426097 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e8416d-4800-46c9-b955-c4f7f76e51c6 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:31.519318 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:31.524859 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e8416d-4800-46c9-b955-c4f7f76e51c6 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:31.557550 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676602b0-c445-4c5b-af30-822456a276a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:31.558720 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676602b0-c445-4c5b-af30-822456a276a3 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:31.558997 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676602b0-c445-4c5b-af30-822456a276a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:31.559268 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676602b0-c445-4c5b-af30-822456a276a3 None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:31.559894 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676602b0-c445-4c5b-af30-822456a276a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:31.560193 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676602b0-c445-4c5b-af30-822456a276a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:31.561575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676602b0-c445-4c5b-af30-822456a276a3 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:31.572177 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 616/1243] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:31.578101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63242cf1-585e-4783-9f9c-fb3bf1b448a9 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:31.656957 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:31.677725 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63242cf1-585e-4783-9f9c-fb3bf1b448a9 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:31.709746 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42817100-f032-47e0-8cb7-d275c7c974df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:31.710368 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42817100-f032-47e0-8cb7-d275c7c974df None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:31.710644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42817100-f032-47e0-8cb7-d275c7c974df None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:31.710915 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42817100-f032-47e0-8cb7-d275c7c974df None admin] PATH_INFO: `/v3/projects/service` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:31.717987 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42817100-f032-47e0-8cb7-d275c7c974df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:31.719420 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42817100-f032-47e0-8cb7-d275c7c974df None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:31.730739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-42817100-f032-47e0-8cb7-d275c7c974df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:38:31.730739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:31.730739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:31.730739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:31.730739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:31.730739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:31.730739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:31.730739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:31.730739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:31.730739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:31.730739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:31.730739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:31.730739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:38:31.730739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:38:31.730739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:38:31.731831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:38:31.731831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:31.731831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:31.731831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:31.731831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:31.731831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:31.731831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:31.731831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:31.731831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:31.731831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:31.731831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:31.731831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:31.731831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:31.731831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:31.731831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:31.731831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:31.731831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:31.733223 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:38:31.733223 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:38:31.733223 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:38:31.733223 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:38:31.733223 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:38:31.733223 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:38:31.733223 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:38:31.733223 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:38:31.733223 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 628/1244] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:38:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:31.737785 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676602b0-c445-4c5b-af30-822456a276a3 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:31.836140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676602b0-c445-4c5b-af30-822456a276a3 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:31.869657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c36322-cf58-42ef-8e57-097b3deb0163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:31.870252 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c36322-cf58-42ef-8e57-097b3deb0163 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:31.870609 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c36322-cf58-42ef-8e57-097b3deb0163 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:31.870879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c36322-cf58-42ef-8e57-097b3deb0163 None admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:31.871568 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c36322-cf58-42ef-8e57-097b3deb0163 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:31.871873 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c36322-cf58-42ef-8e57-097b3deb0163 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:31.873314 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c36322-cf58-42ef-8e57-097b3deb0163 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:31.885195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 617/1245] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:38:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:31.891398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42817100-f032-47e0-8cb7-d275c7c974df None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:31.993014 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42817100-f032-47e0-8cb7-d275c7c974df None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:32.027231 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112468f-b22a-439a-b1bc-5f85b9634ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:32.027838 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112468f-b22a-439a-b1bc-5f85b9634ec6 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:32.028120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112468f-b22a-439a-b1bc-5f85b9634ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:32.028394 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112468f-b22a-439a-b1bc-5f85b9634ec6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:32.029184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112468f-b22a-439a-b1bc-5f85b9634ec6 None admin] RBAC: Adding query filter params (role.id=bce78c7e0a534135a820de7f3588aa4d, scope.project.id=fde7ac3ed472433ab3bba56e5e091510, user.id=2876f4182f0949039b862f053533fe85) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:32.029496 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112468f-b22a-439a-b1bc-5f85b9634ec6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:32.031366 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112468f-b22a-439a-b1bc-5f85b9634ec6 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:32.036816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 629/1246] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:38:31 2022] GET /identity/v3/role_assignments?user.id=2876f4182f0949039b862f053533fe85&scope.project.id=fde7ac3ed472433ab3bba56e5e091510&role.id=bce78c7e0a534135a820de7f3588aa4d => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:34.036220 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2f84b-2570-4d99-bd59-ea071f94034c None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:34.036806 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2f84b-2570-4d99-bd59-ea071f94034c None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:34.036806 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2f84b-2570-4d99-bd59-ea071f94034c None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:34.038135 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 618/1247] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:38:34.044046 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e8ff2-7e6a-44d2-b208-68397fc32f67 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:34.044450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e8ff2-7e6a-44d2-b208-68397fc32f67 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:34.044622 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e8ff2-7e6a-44d2-b208-68397fc32f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:34.050755 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:34.063382 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-c21e8ff2-7e6a-44d2-b208-68397fc32f67 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:38:34.064611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-c21e8ff2-7e6a-44d2-b208-68397fc32f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:38:34.130617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 630/1248] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:38:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 00:38:34.400053 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2f84b-2570-4d99-bd59-ea071f94034c None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:34.418524 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:34.499736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2f84b-2570-4d99-bd59-ea071f94034c None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:34.532715 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd775638-6534-4d95-bc6f-8e10aa38032a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:34.533334 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd775638-6534-4d95-bc6f-8e10aa38032a None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:34.533615 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd775638-6534-4d95-bc6f-8e10aa38032a None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:34.533887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd775638-6534-4d95-bc6f-8e10aa38032a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:34.544186 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd775638-6534-4d95-bc6f-8e10aa38032a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:38:34.548825 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-bd775638-6534-4d95-bc6f-8e10aa38032a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:38:34.548825 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:34.548825 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:34.548825 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:34.548825 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:34.548825 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:34.548825 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:34.548825 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:34.548825 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:34.548825 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:34.548825 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:34.548825 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:34.548825 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:38:34.548825 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:38:34.548825 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:38:34.548825 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:38:34.549964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:38:34.549964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:38:34.549964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:38:34.549964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:38:34.549964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:34.549964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:34.549964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:34.549964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:34.549964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:34.549964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:34.549964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:34.549964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:34.549964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:34.549964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:34.549964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:34.549964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:34.551038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:34.551038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:34.551038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:34.551038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:34.551038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:38:34.551038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:38:34.551038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:38:34.551038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:38:34.551038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:38:34.551038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:38:34.551038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:38:34.551038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:38:34.551038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 619/1249] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:38:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:34.556029 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e8ff2-7e6a-44d2-b208-68397fc32f67 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:34.652143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e8ff2-7e6a-44d2-b208-68397fc32f67 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:34.680926 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:34.686281 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1530ab55-75f1-4338-9442-5290710099d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:34.686886 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530ab55-75f1-4338-9442-5290710099d4 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:34.687165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530ab55-75f1-4338-9442-5290710099d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:34.687440 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530ab55-75f1-4338-9442-5290710099d4 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:34.688100 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1530ab55-75f1-4338-9442-5290710099d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:34.688405 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1530ab55-75f1-4338-9442-5290710099d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:34.689793 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1530ab55-75f1-4338-9442-5290710099d4 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:34.697682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 631/1250] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:38:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:34.703412 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd775638-6534-4d95-bc6f-8e10aa38032a None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:34.778389 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:34.800971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd775638-6534-4d95-bc6f-8e10aa38032a None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:34.833700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4daccb3-3daa-4e08-91d1-22e42d564bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:34.834372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4daccb3-3daa-4e08-91d1-22e42d564bb5 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:34.834693 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4daccb3-3daa-4e08-91d1-22e42d564bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:34.834966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4daccb3-3daa-4e08-91d1-22e42d564bb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:34.845218 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4daccb3-3daa-4e08-91d1-22e42d564bb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:34.846731 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4daccb3-3daa-4e08-91d1-22e42d564bb5 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:34.861059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-e4daccb3-3daa-4e08-91d1-22e42d564bb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:38:34.861059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:34.861059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:34.861059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:34.861059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:34.861059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:34.861059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:34.861059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:34.861059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:34.861059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:34.861059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:34.861059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:34.861059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:38:34.861059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:38:34.861059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:38:34.861059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:38:34.862021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:34.862021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:34.862021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:34.862021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:34.862021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:34.862021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:34.862021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:34.862021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:34.862021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:34.862021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:34.862021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:34.862021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:34.862021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:34.862021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:34.862021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:34.862021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:34.862021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:38:34.862991 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:38:34.862991 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:38:34.862991 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:38:34.863166 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 620/1251] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:38:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:34.868256 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1530ab55-75f1-4338-9442-5290710099d4 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:34.963878 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1530ab55-75f1-4338-9442-5290710099d4 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:34.996789 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6a2ab-882d-4ff8-a041-3165b9caf39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:34.997387 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6a2ab-882d-4ff8-a041-3165b9caf39d None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:34.997661 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6a2ab-882d-4ff8-a041-3165b9caf39d None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:34.997926 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6a2ab-882d-4ff8-a041-3165b9caf39d None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:34.998552 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c6a2ab-882d-4ff8-a041-3165b9caf39d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:34.998851 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c6a2ab-882d-4ff8-a041-3165b9caf39d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:35.000206 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c6a2ab-882d-4ff8-a041-3165b9caf39d None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:35.011379 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 632/1252] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:35.017226 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4daccb3-3daa-4e08-91d1-22e42d564bb5 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:35.117488 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4daccb3-3daa-4e08-91d1-22e42d564bb5 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:35.142386 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:35.150051 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21cbcb-7bfd-4596-915f-534752bd2494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:35.150676 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21cbcb-7bfd-4596-915f-534752bd2494 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:35.150953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21cbcb-7bfd-4596-915f-534752bd2494 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:35.151222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21cbcb-7bfd-4596-915f-534752bd2494 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:35.159194 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21cbcb-7bfd-4596-915f-534752bd2494 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:35.160703 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21cbcb-7bfd-4596-915f-534752bd2494 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:35.171875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-df21cbcb-7bfd-4596-915f-534752bd2494 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 00:38:35.171875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:35.171875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:35.171875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:35.171875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:35.171875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:35.171875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:35.171875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:35.171875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:35.171875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:35.171875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:35.171875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:35.171875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:38:35.171875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:38:35.171875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:38:35.171875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:38:35.173107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:35.173107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:35.173107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:38:35.173107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:38:35.173107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:38:35.173107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:38:35.173107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:35.173107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:35.173107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:35.173107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:35.173107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:35.173107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:35.173107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:35.173107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:35.173107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:35.173107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:35.173107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:35.173107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:35.174396 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:35.174396 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:35.174396 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:38:35.174396 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:38:35.174396 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:38:35.174396 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:38:35.174396 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:38:35.174396 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:38:35.174396 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 00:38:35.174396 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:38:35.174396 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 621/1253] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:38:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:35.178922 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6a2ab-882d-4ff8-a041-3165b9caf39d None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:35.234884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:35.283895 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6a2ab-882d-4ff8-a041-3165b9caf39d None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:35.316927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfb94e-e69c-491b-8093-3aaa8099c587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:35.317540 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfb94e-e69c-491b-8093-3aaa8099c587 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:35.317824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfb94e-e69c-491b-8093-3aaa8099c587 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:35.318099 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfb94e-e69c-491b-8093-3aaa8099c587 None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:35.318825 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dfb94e-e69c-491b-8093-3aaa8099c587 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:35.319129 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dfb94e-e69c-491b-8093-3aaa8099c587 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:35.320569 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dfb94e-e69c-491b-8093-3aaa8099c587 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:35.337635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 633/1254] 10.222.0.59 () {56 vars in 1143 bytes} [Fri Aug 26 00:38:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:35.344105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21cbcb-7bfd-4596-915f-534752bd2494 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:35.443475 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21cbcb-7bfd-4596-915f-534752bd2494 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:35.476603 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2528144-5db0-4bc0-8eb4-474c77660ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:35.477212 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2528144-5db0-4bc0-8eb4-474c77660ac0 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:35.477472 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2528144-5db0-4bc0-8eb4-474c77660ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:35.477758 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2528144-5db0-4bc0-8eb4-474c77660ac0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:35.534512 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:35.546556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2528144-5db0-4bc0-8eb4-474c77660ac0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:35.548021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2528144-5db0-4bc0-8eb4-474c77660ac0 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:35.644224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 622/1255] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:38:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 00:38:35.650155 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfb94e-e69c-491b-8093-3aaa8099c587 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:35.746825 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfb94e-e69c-491b-8093-3aaa8099c587 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:35.760105 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:35.780648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d1e79-b81a-4f29-b55d-8357c46b4d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:35.781268 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d1e79-b81a-4f29-b55d-8357c46b4d3b None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:35.781546 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d1e79-b81a-4f29-b55d-8357c46b4d3b None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:35.781818 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d1e79-b81a-4f29-b55d-8357c46b4d3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:35.791402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d1e79-b81a-4f29-b55d-8357c46b4d3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:35.792842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d1e79-b81a-4f29-b55d-8357c46b4d3b None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:35.806668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-cb8d1e79-b81a-4f29-b55d-8357c46b4d3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:38:35.806668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:35.806668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:35.806668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:35.806668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:35.806668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:35.806668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:35.806668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:35.806668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:35.806668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:35.806668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:35.806668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:35.806668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:38:35.806668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:38:35.806668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:38:35.806668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:38:35.807791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:35.807791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:35.807791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:35.807791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:35.807791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:35.807791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:35.807791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:35.807791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:35.807791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:35.807791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:35.807791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:35.807791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:35.807791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:35.807791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:35.807791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:35.807791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:35.807791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:38:35.809667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:38:35.809667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:38:35.809667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:38:35.809667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 634/1256] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:38:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:38:35.814047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2528144-5db0-4bc0-8eb4-474c77660ac0 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:35.898059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:35.910214 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2528144-5db0-4bc0-8eb4-474c77660ac0 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:35.944090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd4de8-b226-4f98-9460-3196d4b07a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:35.944720 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd4de8-b226-4f98-9460-3196d4b07a23 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:35.945002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd4de8-b226-4f98-9460-3196d4b07a23 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:35.945274 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd4de8-b226-4f98-9460-3196d4b07a23 None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:35.945910 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd4de8-b226-4f98-9460-3196d4b07a23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:35.946213 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd4de8-b226-4f98-9460-3196d4b07a23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:35.947564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd4de8-b226-4f98-9460-3196d4b07a23 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:35.959455 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 623/1257] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:35.965789 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d1e79-b81a-4f29-b55d-8357c46b4d3b None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:36.065679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d1e79-b81a-4f29-b55d-8357c46b4d3b None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:36.097869 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7614ca-3042-4a3d-9f3d-3bfd50999b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:36.098494 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7614ca-3042-4a3d-9f3d-3bfd50999b60 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:36.098774 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7614ca-3042-4a3d-9f3d-3bfd50999b60 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:36.099049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7614ca-3042-4a3d-9f3d-3bfd50999b60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:36.105984 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7614ca-3042-4a3d-9f3d-3bfd50999b60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:36.107419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7614ca-3042-4a3d-9f3d-3bfd50999b60 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:36.118521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-0a7614ca-3042-4a3d-9f3d-3bfd50999b60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:38:36.118521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:36.118521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:36.118521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:36.118521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:36.118521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:36.118521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:36.118521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:36.118521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:36.118521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:36.118521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:36.118521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:36.118521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:38:36.118521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:38:36.118521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:38:36.121623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:38:36.121623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:36.121623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:36.121623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:36.121623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:36.121623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:36.121623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:36.121623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:36.121623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:36.121623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:36.121623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:36.121623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:36.121623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:36.121623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:36.121623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:36.121623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:36.121623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:36.122800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:38:36.122800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:38:36.122800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:38:36.122800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:38:36.122800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:38:36.122800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:38:36.122800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:38:36.122800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:38:36.122800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 635/1258] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:38:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:36.126242 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd4de8-b226-4f98-9460-3196d4b07a23 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:36.224275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd4de8-b226-4f98-9460-3196d4b07a23 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:36.258432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d8030a-e8db-4f58-a904-922f43d79cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:36.259031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8030a-e8db-4f58-a904-922f43d79cae None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:36.259313 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8030a-e8db-4f58-a904-922f43d79cae None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:36.259589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8030a-e8db-4f58-a904-922f43d79cae None admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:36.260281 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d8030a-e8db-4f58-a904-922f43d79cae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:36.260620 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d8030a-e8db-4f58-a904-922f43d79cae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:36.262569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d8030a-e8db-4f58-a904-922f43d79cae None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:36.274038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 624/1259] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:38:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:36.281191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7614ca-3042-4a3d-9f3d-3bfd50999b60 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:36.306714 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:36.380096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7614ca-3042-4a3d-9f3d-3bfd50999b60 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:36.413561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0584e6-84f7-4c35-a8fc-dac0bc58e48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:36.414168 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0584e6-84f7-4c35-a8fc-dac0bc58e48d None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:36.414448 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0584e6-84f7-4c35-a8fc-dac0bc58e48d None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:36.414724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0584e6-84f7-4c35-a8fc-dac0bc58e48d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:36.415444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0584e6-84f7-4c35-a8fc-dac0bc58e48d None admin] RBAC: Adding query filter params (role.id=1a33e3c8cb8347feb9498595886cc621, scope.project.id=fde7ac3ed472433ab3bba56e5e091510, user.id=2876f4182f0949039b862f053533fe85) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:36.415754 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0584e6-84f7-4c35-a8fc-dac0bc58e48d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:36.417222 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0584e6-84f7-4c35-a8fc-dac0bc58e48d None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:36.422848 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 636/1260] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:38:36 2022] GET /identity/v3/role_assignments?user.id=2876f4182f0949039b862f053533fe85&scope.project.id=fde7ac3ed472433ab3bba56e5e091510&role.id=1a33e3c8cb8347feb9498595886cc621 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:38.284829 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb186c-1965-486f-978c-7b3c1b11a6d0 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:38.285382 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb186c-1965-486f-978c-7b3c1b11a6d0 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:38.285382 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb186c-1965-486f-978c-7b3c1b11a6d0 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:38.286783 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 625/1261] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:38:38.291861 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234641ba-c2e3-49df-8f49-9a403b756372 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:38.292309 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234641ba-c2e3-49df-8f49-9a403b756372 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:38.292403 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234641ba-c2e3-49df-8f49-9a403b756372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:38.310643 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-234641ba-c2e3-49df-8f49-9a403b756372 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:38:38.311811 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-234641ba-c2e3-49df-8f49-9a403b756372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:38:38.378601 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 637/1262] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:38:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:38:38.677844 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb186c-1965-486f-978c-7b3c1b11a6d0 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:38.689444 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:38.777770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb186c-1965-486f-978c-7b3c1b11a6d0 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:38.811285 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bac05b-112e-4bab-a780-b5189cb387ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:38.811966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bac05b-112e-4bab-a780-b5189cb387ed None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:38.812261 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bac05b-112e-4bab-a780-b5189cb387ed None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:38.812537 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bac05b-112e-4bab-a780-b5189cb387ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:38.822674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bac05b-112e-4bab-a780-b5189cb387ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:38:38.828048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-09bac05b-112e-4bab-a780-b5189cb387ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:38:38.828048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:38.828048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:38.828048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:38.828048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:38.828048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:38.828048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:38.828048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:38.828048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:38.828048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:38.828048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:38.828048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:38.828048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:38:38.828048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:38:38.828048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:38:38.828048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:38:38.829062 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:38:38.829062 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:38:38.829062 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:38:38.829062 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:38:38.829062 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:38.829062 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:38.829062 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:38.829062 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:38.829062 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:38.829062 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:38.829062 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:38.829062 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:38.829062 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:38.829062 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:38.829062 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:38.829062 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:38.829942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:38.829942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:38.829942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:38.829942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:38.829942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:38:38.829942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:38:38.829942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:38:38.829942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:38:38.829942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:38:38.829942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:38:38.829942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:38:38.829942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:38:38.829942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 626/1263] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:38:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:38.835385 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234641ba-c2e3-49df-8f49-9a403b756372 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:38.853514 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:38.932516 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234641ba-c2e3-49df-8f49-9a403b756372 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:38.966611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62edddba-0067-4866-be6b-f10b73f58417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:38.967213 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62edddba-0067-4866-be6b-f10b73f58417 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:38.967490 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62edddba-0067-4866-be6b-f10b73f58417 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:38.967760 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62edddba-0067-4866-be6b-f10b73f58417 None admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:38.968412 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62edddba-0067-4866-be6b-f10b73f58417 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:38.968738 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62edddba-0067-4866-be6b-f10b73f58417 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:38.970124 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62edddba-0067-4866-be6b-f10b73f58417 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:38.978379 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 638/1264] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:38:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:38.984444 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bac05b-112e-4bab-a780-b5189cb387ed None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:39.051091 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:39.083493 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bac05b-112e-4bab-a780-b5189cb387ed None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:39.117954 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4322e-3a72-4017-b9e6-83a8e5b25b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:39.118726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4322e-3a72-4017-b9e6-83a8e5b25b36 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:39.119008 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4322e-3a72-4017-b9e6-83a8e5b25b36 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:39.119284 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4322e-3a72-4017-b9e6-83a8e5b25b36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:39.129553 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4322e-3a72-4017-b9e6-83a8e5b25b36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:39.131002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4322e-3a72-4017-b9e6-83a8e5b25b36 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:39.145078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-0fb4322e-3a72-4017-b9e6-83a8e5b25b36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:38:39.145078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:39.145078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:39.145078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:39.145078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:39.145078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:39.145078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:39.145078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:39.145078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:39.145078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:39.145078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:39.145078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:39.145078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:38:39.145078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:38:39.145078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:38:39.145078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:38:39.146103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:39.146103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:39.146103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:39.146103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:39.146103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:39.146103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:39.146103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:39.146103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:39.146103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:39.146103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:39.146103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:39.146103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:39.146103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:39.146103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:39.146103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:39.146103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:39.146103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:38:39.147601 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:38:39.147601 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:38:39.147601 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:38:39.147601 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 627/1265] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:38:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:39.151998 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62edddba-0067-4866-be6b-f10b73f58417 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:39.233353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:39.252040 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62edddba-0067-4866-be6b-f10b73f58417 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:39.285342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157d54c-0bfd-452c-9aa2-5c70d3524e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:39.285945 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157d54c-0bfd-452c-9aa2-5c70d3524e77 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:39.286229 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157d54c-0bfd-452c-9aa2-5c70d3524e77 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:39.286498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157d54c-0bfd-452c-9aa2-5c70d3524e77 None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:39.287200 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a157d54c-0bfd-452c-9aa2-5c70d3524e77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:39.287506 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a157d54c-0bfd-452c-9aa2-5c70d3524e77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:39.288899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a157d54c-0bfd-452c-9aa2-5c70d3524e77 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:39.300457 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 639/1266] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:39.306410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4322e-3a72-4017-b9e6-83a8e5b25b36 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:39.407066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4322e-3a72-4017-b9e6-83a8e5b25b36 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:39.426734 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:39.440366 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8483fcf4-9d67-4e6f-99c5-202450dfce13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:39.441024 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483fcf4-9d67-4e6f-99c5-202450dfce13 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:39.441301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483fcf4-9d67-4e6f-99c5-202450dfce13 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:39.441573 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483fcf4-9d67-4e6f-99c5-202450dfce13 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:39.449317 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8483fcf4-9d67-4e6f-99c5-202450dfce13 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:39.451004 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8483fcf4-9d67-4e6f-99c5-202450dfce13 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:39.461832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-8483fcf4-9d67-4e6f-99c5-202450dfce13 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 00:38:39.461832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:39.461832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:39.461832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:39.461832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:39.461832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:39.461832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:39.461832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:39.461832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:39.461832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:39.461832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:39.461832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:39.461832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:38:39.461832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:38:39.461832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:38:39.461832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:38:39.462928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:39.462928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:39.462928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:38:39.462928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:38:39.462928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:38:39.462928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:38:39.462928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:39.462928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:39.462928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:39.462928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:39.462928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:39.462928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:39.462928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:39.462928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:39.462928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:39.462928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:39.462928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:39.462928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:39.463967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:39.463967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:39.463967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:38:39.463967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:38:39.463967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:38:39.463967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:38:39.463967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:38:39.463967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:38:39.463967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 00:38:39.463967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:38:39.463967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 628/1267] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:38:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:39.482980 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157d54c-0bfd-452c-9aa2-5c70d3524e77 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:39.568135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157d54c-0bfd-452c-9aa2-5c70d3524e77 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:39.601624 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b12a6c-7027-41b7-b0e5-cf13946deca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:39.602228 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b12a6c-7027-41b7-b0e5-cf13946deca0 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:39.602508 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b12a6c-7027-41b7-b0e5-cf13946deca0 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:39.602784 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b12a6c-7027-41b7-b0e5-cf13946deca0 None admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:39.603507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b12a6c-7027-41b7-b0e5-cf13946deca0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:39.603813 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b12a6c-7027-41b7-b0e5-cf13946deca0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:39.605395 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b12a6c-7027-41b7-b0e5-cf13946deca0 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:39.614526 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:39.622781 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 640/1268] 10.222.0.59 () {56 vars in 1143 bytes} [Fri Aug 26 00:38:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:39.628427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8483fcf4-9d67-4e6f-99c5-202450dfce13 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:39.725232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8483fcf4-9d67-4e6f-99c5-202450dfce13 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:39.758317 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce71a9-6578-4e18-b216-6de8f727c58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:39.758937 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce71a9-6578-4e18-b216-6de8f727c58f None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:39.759221 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce71a9-6578-4e18-b216-6de8f727c58f None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:39.759495 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce71a9-6578-4e18-b216-6de8f727c58f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:39.771223 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce71a9-6578-4e18-b216-6de8f727c58f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:39.772718 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce71a9-6578-4e18-b216-6de8f727c58f None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:39.786662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-1dce71a9-6578-4e18-b216-6de8f727c58f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:38:39.786662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:39.786662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:39.786662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:39.786662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:39.786662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:39.786662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:39.786662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:39.786662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:39.786662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:39.786662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:39.786662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:39.786662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:38:39.786662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:38:39.786662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:38:39.786662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:38:39.787866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:39.787866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:39.787866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:39.787866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:39.787866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:39.787866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:39.787866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:39.787866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:39.787866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:39.787866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:39.787866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:39.787866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:39.787866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:39.787866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:39.787866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:39.787866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:39.787866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:38:39.791969 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:38:39.791969 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:38:39.791969 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:38:39.791969 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 629/1269] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:38:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:39.794156 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b12a6c-7027-41b7-b0e5-cf13946deca0 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:39.891164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b12a6c-7027-41b7-b0e5-cf13946deca0 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:39.924098 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed6f2b-78e1-45db-bb7e-7a7e49ba193a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:39.924737 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed6f2b-78e1-45db-bb7e-7a7e49ba193a None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:39.925017 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed6f2b-78e1-45db-bb7e-7a7e49ba193a None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:39.925286 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed6f2b-78e1-45db-bb7e-7a7e49ba193a None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:39.925918 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed6f2b-78e1-45db-bb7e-7a7e49ba193a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:39.926218 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed6f2b-78e1-45db-bb7e-7a7e49ba193a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:39.927567 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed6f2b-78e1-45db-bb7e-7a7e49ba193a None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:39.939095 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 641/1270] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:39.945118 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce71a9-6578-4e18-b216-6de8f727c58f None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:39.975287 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:40.040061 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce71a9-6578-4e18-b216-6de8f727c58f None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:40.076972 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9cf0fc-b8de-4e89-b0af-626fab67d224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:40.077598 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9cf0fc-b8de-4e89-b0af-626fab67d224 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:40.077944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9cf0fc-b8de-4e89-b0af-626fab67d224 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:40.078228 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9cf0fc-b8de-4e89-b0af-626fab67d224 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:40.084977 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9cf0fc-b8de-4e89-b0af-626fab67d224 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:40.086477 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9cf0fc-b8de-4e89-b0af-626fab67d224 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:40.098588 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-cc9cf0fc-b8de-4e89-b0af-626fab67d224 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:38:40.098588 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:40.098588 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:40.098588 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:40.098588 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:40.098588 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:40.098588 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:40.098588 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:40.098588 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:40.098588 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:40.098588 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:40.098588 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:40.098588 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:38:40.098588 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:38:40.098588 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:38:40.099638 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:38:40.099638 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:40.099638 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:40.099638 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:40.099638 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:40.099638 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:40.099638 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:40.099638 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:40.099638 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:40.099638 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:40.099638 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:40.099638 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:40.099638 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:40.099638 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:40.099638 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:40.099638 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:40.099638 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:40.100780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:38:40.100780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:38:40.100780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:38:40.100780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:38:40.100780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:38:40.100780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:38:40.100780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:38:40.100780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:38:40.100780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 630/1271] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:38:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:38:40.106291 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed6f2b-78e1-45db-bb7e-7a7e49ba193a None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:40.163653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:40.205006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed6f2b-78e1-45db-bb7e-7a7e49ba193a None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:40.238359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8532615-ea5e-4fcf-a4bf-c81be7ad5c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:40.238963 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8532615-ea5e-4fcf-a4bf-c81be7ad5c53 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:40.239243 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8532615-ea5e-4fcf-a4bf-c81be7ad5c53 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:40.239517 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8532615-ea5e-4fcf-a4bf-c81be7ad5c53 None admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:40.240195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8532615-ea5e-4fcf-a4bf-c81be7ad5c53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:40.240528 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8532615-ea5e-4fcf-a4bf-c81be7ad5c53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:40.241951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8532615-ea5e-4fcf-a4bf-c81be7ad5c53 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:40.253464 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 642/1272] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:38:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:40.259981 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9cf0fc-b8de-4e89-b0af-626fab67d224 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:40.350780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:40.359633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9cf0fc-b8de-4e89-b0af-626fab67d224 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:40.392266 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a5be83-9659-4dbb-a5d1-5fca1b77bcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:40.392936 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5be83-9659-4dbb-a5d1-5fca1b77bcca None admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:40.393212 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5be83-9659-4dbb-a5d1-5fca1b77bcca None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:40.393481 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5be83-9659-4dbb-a5d1-5fca1b77bcca None admin] PATH_INFO: `/v3/projects/fde7ac3ed472433ab3bba56e5e091510/users/2876f4182f0949039b862f053533fe85/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:40.395344 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a5be83-9659-4dbb-a5d1-5fca1b77bcca None admin] RBAC: Authorizing `identity:create_grant(project_id=fde7ac3ed472433ab3bba56e5e091510, user_id=2876f4182f0949039b862f053533fe85, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:40.396787 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a5be83-9659-4dbb-a5d1-5fca1b77bcca None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:40.488073 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 631/1273] 10.222.0.59 () {58 vars in 1439 bytes} [Fri Aug 26 00:38:40 2022] PUT /identity/v3/projects/fde7ac3ed472433ab3bba56e5e091510/users/2876f4182f0949039b862f053533fe85/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:38:42.391167 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373941f-7d82-4ca0-bed6-59911eb58d4f None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:42.391729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373941f-7d82-4ca0-bed6-59911eb58d4f None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:42.391729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373941f-7d82-4ca0-bed6-59911eb58d4f None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:42.393116 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 643/1274] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:38:42.398529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241d5a8-2bec-46fe-9f7e-cc92eb170c25 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:42.398952 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241d5a8-2bec-46fe-9f7e-cc92eb170c25 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:42.399097 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241d5a8-2bec-46fe-9f7e-cc92eb170c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:42.419280 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-8241d5a8-2bec-46fe-9f7e-cc92eb170c25 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:38:42.420516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-8241d5a8-2bec-46fe-9f7e-cc92eb170c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:38:42.489592 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 632/1275] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:38:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:38:42.762822 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373941f-7d82-4ca0-bed6-59911eb58d4f None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:42.862238 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373941f-7d82-4ca0-bed6-59911eb58d4f None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:42.879834 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:42.896336 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc554a-d563-4eb6-8b6a-0acd53ac4248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:42.896961 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc554a-d563-4eb6-8b6a-0acd53ac4248 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:42.897244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc554a-d563-4eb6-8b6a-0acd53ac4248 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:42.897518 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc554a-d563-4eb6-8b6a-0acd53ac4248 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:42.908306 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc554a-d563-4eb6-8b6a-0acd53ac4248 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 00:38:42.913018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-e3cc554a-d563-4eb6-8b6a-0acd53ac4248 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:38:42.913018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:42.913018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:42.913018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:42.913018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:42.913018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:42.913018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:42.913018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:42.913018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:42.913018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:42.913018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:42.913018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:42.913018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 00:38:42.913018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 00:38:42.913018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 00:38:42.913018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 00:38:42.914015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 00:38:42.914015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 00:38:42.914015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 00:38:42.914015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 00:38:42.914015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:42.914015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:42.914015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:42.914015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:42.914015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:42.914015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:42.914015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:42.914015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:42.914015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:42.914015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:42.914015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:42.914015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:42.914928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:42.914928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:42.914928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:42.914928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:42.914928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 00:38:42.914928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 00:38:42.914928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 00:38:42.914928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 00:38:42.914928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 00:38:42.914928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 00:38:42.914928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 00:38:42.914928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:38:42.914928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 644/1276] 10.222.0.59 () {56 vars in 1096 bytes} [Fri Aug 26 00:38:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:42.920227 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241d5a8-2bec-46fe-9f7e-cc92eb170c25 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:42.999619 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:43.023107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241d5a8-2bec-46fe-9f7e-cc92eb170c25 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:43.049922 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe598f-95be-442d-b9c7-c73293c200d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:43.050525 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe598f-95be-442d-b9c7-c73293c200d9 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:43.050803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe598f-95be-442d-b9c7-c73293c200d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:43.051075 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe598f-95be-442d-b9c7-c73293c200d9 None admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:43.051767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe598f-95be-442d-b9c7-c73293c200d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:43.052062 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe598f-95be-442d-b9c7-c73293c200d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:43.053448 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe598f-95be-442d-b9c7-c73293c200d9 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:43.061145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 633/1277] 10.222.0.59 () {56 vars in 1099 bytes} [Fri Aug 26 00:38:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:43.066906 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc554a-d563-4eb6-8b6a-0acd53ac4248 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:43.165539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc554a-d563-4eb6-8b6a-0acd53ac4248 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:43.199980 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bfa13d-2222-43b3-8a18-8233e8f56811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:43.200643 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfa13d-2222-43b3-8a18-8233e8f56811 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:43.200926 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfa13d-2222-43b3-8a18-8233e8f56811 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:43.201203 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfa13d-2222-43b3-8a18-8233e8f56811 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:43.211047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bfa13d-2222-43b3-8a18-8233e8f56811 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:43.212502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bfa13d-2222-43b3-8a18-8233e8f56811 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:43.226914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-73bfa13d-2222-43b3-8a18-8233e8f56811 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:38:43.226914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:43.226914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:43.226914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:43.226914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:43.226914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:43.226914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:43.226914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:43.226914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:43.226914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:43.226914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:43.226914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:43.226914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:38:43.226914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:38:43.226914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:38:43.226914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:38:43.228090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:43.228090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:43.228090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:43.228090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:43.228090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:43.228090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:43.228090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:43.228090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:43.228090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:43.228090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:43.228090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:43.228090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:43.228090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:43.228090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:43.228090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:43.228090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:43.228090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:38:43.229408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:38:43.229408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:38:43.229408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:38:43.229408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 645/1278] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:38:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:43.234009 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe598f-95be-442d-b9c7-c73293c200d9 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:43.331706 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe598f-95be-442d-b9c7-c73293c200d9 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:43.363982 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07617afc-ef8b-4d1d-a232-e68ac73e6013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:43.364613 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07617afc-ef8b-4d1d-a232-e68ac73e6013 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:43.364897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07617afc-ef8b-4d1d-a232-e68ac73e6013 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:43.365175 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07617afc-ef8b-4d1d-a232-e68ac73e6013 None admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:43.365856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07617afc-ef8b-4d1d-a232-e68ac73e6013 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:43.366193 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07617afc-ef8b-4d1d-a232-e68ac73e6013 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:43.367559 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07617afc-ef8b-4d1d-a232-e68ac73e6013 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:43.379724 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 634/1279] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:38:43.385723 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bfa13d-2222-43b3-8a18-8233e8f56811 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:43.417328 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:43.485651 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bfa13d-2222-43b3-8a18-8233e8f56811 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:43.518517 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f3a49-f3f8-4881-926a-3aaf7b80848d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:43.519140 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f3a49-f3f8-4881-926a-3aaf7b80848d None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:43.519415 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f3a49-f3f8-4881-926a-3aaf7b80848d None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:43.519687 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f3a49-f3f8-4881-926a-3aaf7b80848d None admin] PATH_INFO: `/v3/users/placement` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:43.527274 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f3a49-f3f8-4881-926a-3aaf7b80848d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:43.528734 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f3a49-f3f8-4881-926a-3aaf7b80848d None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:43.540265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-b56f3a49-f3f8-4881-926a-3aaf7b80848d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 00:38:43.540265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:43.540265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:43.540265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:43.540265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:43.540265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:43.540265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:43.540265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:43.540265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:43.540265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:43.540265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:43.540265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:43.540265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 00:38:43.540265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 00:38:43.540265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 00:38:43.540265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 00:38:43.541447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:43.541447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:43.541447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 00:38:43.541447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:38:43.541447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 00:38:43.541447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 00:38:43.541447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:43.541447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:43.541447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:43.541447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:43.541447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:43.541447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:43.541447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:43.541447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:43.541447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:43.541447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:43.541447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:43.541447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:43.543224 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:43.543224 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:43.543224 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 00:38:43.543224 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 00:38:43.543224 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 00:38:43.543224 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 00:38:43.543224 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 00:38:43.543224 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 00:38:43.543224 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 00:38:43.543224 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:38:43.543224 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 646/1280] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:38:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:43.547961 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07617afc-ef8b-4d1d-a232-e68ac73e6013 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:43.551812 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:43.646061 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07617afc-ef8b-4d1d-a232-e68ac73e6013 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:43.678763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b50f81-93a1-47e1-991c-1842962b4d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:43.679491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b50f81-93a1-47e1-991c-1842962b4d11 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:43.679773 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b50f81-93a1-47e1-991c-1842962b4d11 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:43.680049 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b50f81-93a1-47e1-991c-1842962b4d11 None admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:43.680797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b50f81-93a1-47e1-991c-1842962b4d11 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:43.681103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b50f81-93a1-47e1-991c-1842962b4d11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:43.682536 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b50f81-93a1-47e1-991c-1842962b4d11 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:43.701498 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 635/1281] 10.222.0.59 () {56 vars in 1143 bytes} [Fri Aug 26 00:38:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:43.707446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f3a49-f3f8-4881-926a-3aaf7b80848d None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:43.797853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:43.806834 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f3a49-f3f8-4881-926a-3aaf7b80848d None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:43.840721 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47b60c-818b-4365-b8fa-7a935bc970be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:43.841330 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47b60c-818b-4365-b8fa-7a935bc970be None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:43.841611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47b60c-818b-4365-b8fa-7a935bc970be None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:43.841887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47b60c-818b-4365-b8fa-7a935bc970be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:43.909172 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47b60c-818b-4365-b8fa-7a935bc970be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:43.910557 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47b60c-818b-4365-b8fa-7a935bc970be None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:43.996876 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:44.006757 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 647/1282] 10.222.0.59 () {58 vars in 1282 bytes} [Fri Aug 26 00:38:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 00:38:44.012739 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b50f81-93a1-47e1-991c-1842962b4d11 None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:44.072632 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:44.110281 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b50f81-93a1-47e1-991c-1842962b4d11 None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:44.141956 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d714877-e3de-4417-8dfe-a7be3ac587dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:44.142587 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d714877-e3de-4417-8dfe-a7be3ac587dd None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:44.142869 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d714877-e3de-4417-8dfe-a7be3ac587dd None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:44.143143 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d714877-e3de-4417-8dfe-a7be3ac587dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:44.154503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d714877-e3de-4417-8dfe-a7be3ac587dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:44.156198 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d714877-e3de-4417-8dfe-a7be3ac587dd None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:44.170551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-7d714877-e3de-4417-8dfe-a7be3ac587dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:38:44.170551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:44.170551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:44.170551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:44.170551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:44.170551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:44.170551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:44.170551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:44.170551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:44.170551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:44.170551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:44.170551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:44.170551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 00:38:44.170551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 00:38:44.170551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 00:38:44.170551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 00:38:44.171854 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:44.171854 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:44.171854 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:44.171854 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:44.171854 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:44.171854 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:44.171854 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:44.171854 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:44.171854 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:44.171854 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:44.171854 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:44.171854 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:44.171854 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:44.171854 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:44.171854 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:44.171854 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:44.171854 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 00:38:44.176395 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 00:38:44.176395 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 00:38:44.176395 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:38:44.176395 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 636/1283] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:38:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 00:38:44.177385 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47b60c-818b-4365-b8fa-7a935bc970be None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:44.274653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47b60c-818b-4365-b8fa-7a935bc970be None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:44.307852 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c8bb6-7e0c-4bf0-a310-754d9b7edfb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:44.308449 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c8bb6-7e0c-4bf0-a310-754d9b7edfb6 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:44.308753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c8bb6-7e0c-4bf0-a310-754d9b7edfb6 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:44.309023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c8bb6-7e0c-4bf0-a310-754d9b7edfb6 None admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:44.309650 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c8bb6-7e0c-4bf0-a310-754d9b7edfb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:44.309947 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c8bb6-7e0c-4bf0-a310-754d9b7edfb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:44.311293 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c8bb6-7e0c-4bf0-a310-754d9b7edfb6 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:44.322050 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 648/1284] 10.222.0.59 () {56 vars in 1109 bytes} [Fri Aug 26 00:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:44.328481 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d714877-e3de-4417-8dfe-a7be3ac587dd None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:44.426603 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d714877-e3de-4417-8dfe-a7be3ac587dd None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:44.440231 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:44.460357 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76a82f-9b59-4458-a326-760cad61eb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:44.461057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76a82f-9b59-4458-a326-760cad61eb8e None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:44.461335 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76a82f-9b59-4458-a326-760cad61eb8e None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:44.461608 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76a82f-9b59-4458-a326-760cad61eb8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:44.468580 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf76a82f-9b59-4458-a326-760cad61eb8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:44.470017 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf76a82f-9b59-4458-a326-760cad61eb8e None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:44.480998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application [None req-cf76a82f-9b59-4458-a326-760cad61eb8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:38:44.480998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:44.480998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:44.480998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:44.480998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:44.480998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:44.480998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:44.480998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:44.480998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:44.480998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:44.480998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:44.480998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:44.480998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 00:38:44.480998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 00:38:44.480998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 00:38:44.482083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 00:38:44.482083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:44.482083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:44.482083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:44.482083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:44.482083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:44.482083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:44.482083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:44.482083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:44.482083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:44.482083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:44.482083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:44.482083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:44.482083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:44.482083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:44.482083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:44.482083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:44.483272 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 00:38:44.483272 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 00:38:44.483272 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 00:38:44.483272 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 00:38:44.483272 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 00:38:44.483272 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 00:38:44.483272 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 00:38:44.483272 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: ERROR keystone.server.flask.application Aug 26 00:38:44.483272 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 637/1285] 10.222.0.59 () {56 vars in 1111 bytes} [Fri Aug 26 00:38:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:44.488244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c8bb6-7e0c-4bf0-a310-754d9b7edfb6 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:44.546575 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:44.595765 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c8bb6-7e0c-4bf0-a310-754d9b7edfb6 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:44.628555 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247e1b1-4a97-48bf-ac90-2117fbd5d8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:44.629159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247e1b1-4a97-48bf-ac90-2117fbd5d8a3 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:44.629439 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247e1b1-4a97-48bf-ac90-2117fbd5d8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:44.629714 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247e1b1-4a97-48bf-ac90-2117fbd5d8a3 None admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:44.630402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247e1b1-4a97-48bf-ac90-2117fbd5d8a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:44.630709 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247e1b1-4a97-48bf-ac90-2117fbd5d8a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:44.632244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247e1b1-4a97-48bf-ac90-2117fbd5d8a3 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:44.645391 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 649/1286] 10.222.0.59 () {56 vars in 1148 bytes} [Fri Aug 26 00:38:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:44.651623 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76a82f-9b59-4458-a326-760cad61eb8e None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:44.751722 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76a82f-9b59-4458-a326-760cad61eb8e None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:44.785100 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebcdc4-072d-4cd4-a41a-324871efb6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:44.785696 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebcdc4-072d-4cd4-a41a-324871efb6b2 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:44.785973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebcdc4-072d-4cd4-a41a-324871efb6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:44.786282 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebcdc4-072d-4cd4-a41a-324871efb6b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:44.787029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ebcdc4-072d-4cd4-a41a-324871efb6b2 None admin] RBAC: Adding query filter params (role.id=1a33e3c8cb8347feb9498595886cc621, scope.project.id=fde7ac3ed472433ab3bba56e5e091510, user.id=2876f4182f0949039b862f053533fe85) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:44.787333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ebcdc4-072d-4cd4-a41a-324871efb6b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:44.788781 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ebcdc4-072d-4cd4-a41a-324871efb6b2 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:44.794807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 638/1287] 10.222.0.59 () {56 vars in 1374 bytes} [Fri Aug 26 00:38:44 2022] GET /identity/v3/role_assignments?user.id=2876f4182f0949039b862f053533fe85&scope.project.id=fde7ac3ed472433ab3bba56e5e091510&role.id=1a33e3c8cb8347feb9498595886cc621 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:46.734809 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce1fcd-6894-4bec-a4a1-f75776ceaaed None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:46.735343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce1fcd-6894-4bec-a4a1-f75776ceaaed None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:46.735343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce1fcd-6894-4bec-a4a1-f75776ceaaed None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:46.736624 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 650/1288] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:38:46.741893 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc2a9a-a0e0-4434-a47b-c0166806fe2e None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:46.743924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc2a9a-a0e0-4434-a47b-c0166806fe2e None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:46.743924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc2a9a-a0e0-4434-a47b-c0166806fe2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:46.760796 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-f1cc2a9a-a0e0-4434-a47b-c0166806fe2e None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:38:46.761997 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-f1cc2a9a-a0e0-4434-a47b-c0166806fe2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:38:46.790686 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:46.826415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 639/1289] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:38:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 00:38:47.094340 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce1fcd-6894-4bec-a4a1-f75776ceaaed None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:47.193712 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce1fcd-6894-4bec-a4a1-f75776ceaaed None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:47.204799 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:47.227427 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c710f2-f54d-46dc-825e-a61c69bd2f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:47.228046 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c710f2-f54d-46dc-825e-a61c69bd2f63 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:47.228314 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c710f2-f54d-46dc-825e-a61c69bd2f63 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:47.228615 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c710f2-f54d-46dc-825e-a61c69bd2f63 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:47.229187 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c710f2-f54d-46dc-825e-a61c69bd2f63 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:47.230598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c710f2-f54d-46dc-825e-a61c69bd2f63 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:47.239034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-35c710f2-f54d-46dc-825e-a61c69bd2f63 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 00:38:47.239034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:47.239034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 00:38:47.239034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 00:38:47.239034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 00:38:47.239034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 00:38:47.239034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 00:38:47.239034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:38:47.239034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 00:38:47.239034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:38:47.239034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:38:47.239034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:38:47.239034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:47.239034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:47.239034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:47.239034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:47.239034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:47.240298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:47.240298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:47.240298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:47.240298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:47.240298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:47.240298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:47.240298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 00:38:47.240298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 00:38:47.240298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 00:38:47.240298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 00:38:47.240298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:47.240298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:47.240298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:47.240298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:47.240298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:47.240298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:47.241483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:47.241483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:47.241483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:47.241483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:47.241483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:47.241483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:47.241483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:47.241483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:47.241483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:47.241483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:47.241483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 00:38:47.241483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:38:47.241483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 00:38:47.241483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:38:47.241483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 651/1290] 10.222.0.59 () {56 vars in 1117 bytes} [Fri Aug 26 00:38:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:47.246246 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc2a9a-a0e0-4434-a47b-c0166806fe2e None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:47.342810 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc2a9a-a0e0-4434-a47b-c0166806fe2e None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:47.375780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca8b486-feb8-4e06-a740-cbfab624152a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:47.376311 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8b486-feb8-4e06-a740-cbfab624152a None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:47.376613 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8b486-feb8-4e06-a740-cbfab624152a None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:47.376882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8b486-feb8-4e06-a740-cbfab624152a None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:47.377517 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca8b486-feb8-4e06-a740-cbfab624152a None admin] RBAC: Adding query filter params (name=placement) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:47.377833 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca8b486-feb8-4e06-a740-cbfab624152a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:47.379169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca8b486-feb8-4e06-a740-cbfab624152a None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:47.383813 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 640/1291] 10.222.0.59 () {56 vars in 1116 bytes} [Fri Aug 26 00:38:47 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:47.390083 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c710f2-f54d-46dc-825e-a61c69bd2f63 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:47.490771 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c710f2-f54d-46dc-825e-a61c69bd2f63 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:47.525066 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faabc6d2-fe44-4be3-bd3d-64c694970e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:47.525670 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabc6d2-fe44-4be3-bd3d-64c694970e25 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:47.525946 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabc6d2-fe44-4be3-bd3d-64c694970e25 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:47.526216 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabc6d2-fe44-4be3-bd3d-64c694970e25 None admin] PATH_INFO: `/v3/services` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:47.527406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faabc6d2-fe44-4be3-bd3d-64c694970e25 None admin] RBAC: Adding query filter params (type=placement) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:47.527709 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faabc6d2-fe44-4be3-bd3d-64c694970e25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:47.529194 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faabc6d2-fe44-4be3-bd3d-64c694970e25 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:47.533837 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 652/1292] 10.222.0.59 () {56 vars in 1116 bytes} [Fri Aug 26 00:38:47 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:49.418403 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f2155-914d-4250-8672-05146f5d7d58 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:49.418947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f2155-914d-4250-8672-05146f5d7d58 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:49.418947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f2155-914d-4250-8672-05146f5d7d58 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:49.420289 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 641/1293] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:38:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:38:49.425411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416537d-1625-4472-8d66-9eb800341cdc None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:49.425818 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416537d-1625-4472-8d66-9eb800341cdc None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:49.425959 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416537d-1625-4472-8d66-9eb800341cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:49.443677 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-a416537d-1625-4472-8d66-9eb800341cdc None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:38:49.444893 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a416537d-1625-4472-8d66-9eb800341cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:38:49.474582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:49.510467 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 653/1294] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:38:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:38:49.780127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f2155-914d-4250-8672-05146f5d7d58 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:49.810643 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:49.878883 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f2155-914d-4250-8672-05146f5d7d58 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:49.911940 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e1aa0-f49d-443b-aea4-24cd709991c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:49.912559 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e1aa0-f49d-443b-aea4-24cd709991c1 None admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:49.912839 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e1aa0-f49d-443b-aea4-24cd709991c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:49.913109 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e1aa0-f49d-443b-aea4-24cd709991c1 None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:49.913753 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e1aa0-f49d-443b-aea4-24cd709991c1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:49.915162 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e1aa0-f49d-443b-aea4-24cd709991c1 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:49.973286 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 642/1295] 10.222.0.59 () {60 vars in 1141 bytes} [Fri Aug 26 00:38:49 2022] POST /identity/v3/services => generated 246 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:38:51.885317 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c927e-76fa-4529-8885-b0697a3974dc None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:51.886367 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c927e-76fa-4529-8885-b0697a3974dc None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:51.886367 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c927e-76fa-4529-8885-b0697a3974dc None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:51.887695 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 654/1296] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:38:51.893025 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ad798-192a-46e9-bf7d-7097823bee72 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:51.893442 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ad798-192a-46e9-bf7d-7097823bee72 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:51.893581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ad798-192a-46e9-bf7d-7097823bee72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:51.912201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-dd6ad798-192a-46e9-bf7d-7097823bee72 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:38:51.913414 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-dd6ad798-192a-46e9-bf7d-7097823bee72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:38:51.983885 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 643/1297] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:38:51 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:38:52.261392 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c927e-76fa-4529-8885-b0697a3974dc None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:52.361795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c927e-76fa-4529-8885-b0697a3974dc None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:52.420575 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072ed1b-3dd2-4cb8-abb5-58cd72ca4fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:52.421218 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072ed1b-3dd2-4cb8-abb5-58cd72ca4fa0 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:52.421536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072ed1b-3dd2-4cb8-abb5-58cd72ca4fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:52.421806 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072ed1b-3dd2-4cb8-abb5-58cd72ca4fa0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:52.422394 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5072ed1b-3dd2-4cb8-abb5-58cd72ca4fa0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:52.423780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5072ed1b-3dd2-4cb8-abb5-58cd72ca4fa0 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:52.432463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-5072ed1b-3dd2-4cb8-abb5-58cd72ca4fa0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 00:38:52.432463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:52.432463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 00:38:52.432463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 00:38:52.432463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 00:38:52.432463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 00:38:52.432463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 00:38:52.432463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:38:52.432463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 00:38:52.432463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:38:52.432463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:38:52.432463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:38:52.432463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:52.432463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:52.432463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:52.432463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:52.432463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:52.434290 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:52.434290 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:52.434290 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:52.434290 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:52.434290 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:52.434290 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:52.434290 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 00:38:52.434290 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 00:38:52.434290 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 00:38:52.434290 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 00:38:52.434290 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:52.434290 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:52.434290 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:52.434290 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:52.434290 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:52.434290 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:52.435289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:52.435289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:52.435289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:52.435289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:52.435289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:52.435289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:52.435289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:52.435289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:52.435289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:52.435289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:52.435289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 00:38:52.435289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:38:52.435289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 00:38:52.435289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:38:52.435289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 655/1298] 10.222.0.59 () {56 vars in 1117 bytes} [Fri Aug 26 00:38:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:52.439618 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ad798-192a-46e9-bf7d-7097823bee72 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:52.457821 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:52.535413 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ad798-192a-46e9-bf7d-7097823bee72 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:52.567644 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ddd22b-1074-4eca-8af2-334ddec149cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:52.568239 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddd22b-1074-4eca-8af2-334ddec149cd None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:52.568570 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddd22b-1074-4eca-8af2-334ddec149cd None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:52.568844 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddd22b-1074-4eca-8af2-334ddec149cd None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:52.569468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ddd22b-1074-4eca-8af2-334ddec149cd None admin] RBAC: Adding query filter params (name=placement) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:52.569768 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ddd22b-1074-4eca-8af2-334ddec149cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:52.571117 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ddd22b-1074-4eca-8af2-334ddec149cd None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:52.576818 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 644/1299] 10.222.0.59 () {56 vars in 1116 bytes} [Fri Aug 26 00:38:52 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:52.582754 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072ed1b-3dd2-4cb8-abb5-58cd72ca4fa0 None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:52.614288 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:52.681773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072ed1b-3dd2-4cb8-abb5-58cd72ca4fa0 None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:52.715784 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedcf52f-852c-41f8-888e-5d503430f245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:52.716381 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedcf52f-852c-41f8-888e-5d503430f245 None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:52.716689 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedcf52f-852c-41f8-888e-5d503430f245 None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:52.716963 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedcf52f-852c-41f8-888e-5d503430f245 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:52.717653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedcf52f-852c-41f8-888e-5d503430f245 None admin] RBAC: Adding query filter params (interface=public, service_id=29dde31a70684f4bb35ffd68221b041c, region_id=RegionOne) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:52.717962 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedcf52f-852c-41f8-888e-5d503430f245 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:52.719389 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedcf52f-852c-41f8-888e-5d503430f245 None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:52.724317 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 656/1300] 10.222.0.59 () {56 vars in 1251 bytes} [Fri Aug 26 00:38:52 2022] GET /identity/v3/endpoints?service_id=29dde31a70684f4bb35ffd68221b041c&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:38:52.730119 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ddd22b-1074-4eca-8af2-334ddec149cd None admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:52.809116 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:52.826123 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ddd22b-1074-4eca-8af2-334ddec149cd None admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:52.858459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a031a1e-7ae2-41e8-9189-9e6763c73321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:52.859065 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a031a1e-7ae2-41e8-9189-9e6763c73321 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:52.859339 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a031a1e-7ae2-41e8-9189-9e6763c73321 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:52.859609 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a031a1e-7ae2-41e8-9189-9e6763c73321 None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:52.860203 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a031a1e-7ae2-41e8-9189-9e6763c73321 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:52.861583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a031a1e-7ae2-41e8-9189-9e6763c73321 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:52.868041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 645/1301] 10.222.0.59 () {56 vars in 1087 bytes} [Fri Aug 26 00:38:52 2022] GET /identity/v3/services => generated 2191 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:38:54.757492 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6b62e-20eb-4010-a9a3-d40b729070bf None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:54.757998 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6b62e-20eb-4010-a9a3-d40b729070bf None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:54.757998 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6b62e-20eb-4010-a9a3-d40b729070bf None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:54.759338 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 657/1302] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:38:54.764974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2275599-2a66-4cb4-831f-a3da7d3e5a90 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:54.765417 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2275599-2a66-4cb4-831f-a3da7d3e5a90 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:54.765513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2275599-2a66-4cb4-831f-a3da7d3e5a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:54.784179 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-c2275599-2a66-4cb4-831f-a3da7d3e5a90 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:38:54.789685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-c2275599-2a66-4cb4-831f-a3da7d3e5a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:38:54.850317 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 646/1303] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:38:54 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 00:38:55.121302 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6b62e-20eb-4010-a9a3-d40b729070bf None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:55.214875 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:55.221543 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6b62e-20eb-4010-a9a3-d40b729070bf None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:55.254873 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe63129-f2dc-49c4-ba18-125ee57fa1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:55.255498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe63129-f2dc-49c4-ba18-125ee57fa1ee None admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:55.255767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe63129-f2dc-49c4-ba18-125ee57fa1ee None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:55.256036 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe63129-f2dc-49c4-ba18-125ee57fa1ee None admin] PATH_INFO: `/v3/services/placement` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:55.256650 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe63129-f2dc-49c4-ba18-125ee57fa1ee None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:55.258077 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe63129-f2dc-49c4-ba18-125ee57fa1ee None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:55.266455 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application [None req-fbe63129-f2dc-49c4-ba18-125ee57fa1ee None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 00:38:55.266455 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:55.266455 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 00:38:55.266455 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 00:38:55.266455 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 00:38:55.266455 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 00:38:55.266455 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 00:38:55.266455 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:38:55.266455 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 00:38:55.266455 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:38:55.266455 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 00:38:55.266455 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:38:55.266455 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 00:38:55.266455 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 00:38:55.266455 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 00:38:55.266455 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 00:38:55.266455 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 00:38:55.267682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 00:38:55.267682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 00:38:55.267682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 00:38:55.267682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 00:38:55.267682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 00:38:55.267682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 00:38:55.267682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 00:38:55.267682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 00:38:55.267682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 00:38:55.267682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 00:38:55.267682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 00:38:55.267682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 00:38:55.267682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 00:38:55.267682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 00:38:55.267682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 00:38:55.267682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 00:38:55.268855 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 00:38:55.268855 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application with Lock( Aug 26 00:38:55.268855 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 00:38:55.268855 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self._enter() Aug 26 00:38:55.268855 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 00:38:55.268855 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 00:38:55.268855 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 00:38:55.268855 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application return self.creator() Aug 26 00:38:55.268855 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 00:38:55.268855 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application created_value = creator( Aug 26 00:38:55.268855 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 00:38:55.268855 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 00:38:55.268855 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 00:38:55.268855 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: ERROR keystone.server.flask.application Aug 26 00:38:55.268855 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 658/1304] 10.222.0.59 () {56 vars in 1117 bytes} [Fri Aug 26 00:38:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 00:38:55.273516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2275599-2a66-4cb4-831f-a3da7d3e5a90 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:55.347837 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:38:55.369776 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2275599-2a66-4cb4-831f-a3da7d3e5a90 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:55.403630 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d45c0b-98bd-47ff-b226-47bf63784801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:55.404170 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d45c0b-98bd-47ff-b226-47bf63784801 None admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:55.404445 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d45c0b-98bd-47ff-b226-47bf63784801 None admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:55.404740 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d45c0b-98bd-47ff-b226-47bf63784801 None admin] PATH_INFO: `/v3/services` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:55.405366 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d45c0b-98bd-47ff-b226-47bf63784801 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:38:55.405936 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d45c0b-98bd-47ff-b226-47bf63784801 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:55.407385 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d45c0b-98bd-47ff-b226-47bf63784801 None admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:55.412876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 647/1305] 10.222.0.59 () {56 vars in 1116 bytes} [Fri Aug 26 00:38:55 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:38:55.422752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe63129-f2dc-49c4-ba18-125ee57fa1ee None admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:38:55.523050 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe63129-f2dc-49c4-ba18-125ee57fa1ee None admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:38:55.556034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aee377-a99d-4749-a2d9-13643f3647dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:38:55.556668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aee377-a99d-4749-a2d9-13643f3647dc None admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:38:55.556952 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aee377-a99d-4749-a2d9-13643f3647dc None admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:38:55.557226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aee377-a99d-4749-a2d9-13643f3647dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:38:55.557886 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aee377-a99d-4749-a2d9-13643f3647dc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:38:55.559250 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aee377-a99d-4749-a2d9-13643f3647dc None admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:38:55.669895 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 659/1306] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:38:55 2022] POST /identity/v3/endpoints => generated 331 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:39:05.409719 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=30 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:39:05.665529 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:45:33.752960 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0e4b9-ad5f-4642-a14e-bde9ddb07f30 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:33.753508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0e4b9-ad5f-4642-a14e-bde9ddb07f30 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:33.753508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0e4b9-ad5f-4642-a14e-bde9ddb07f30 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:33.756083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 648/1307] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:45:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:45:33.760664 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abb630-dbae-4d67-8476-c6ebf5413d06 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:33.761444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abb630-dbae-4d67-8476-c6ebf5413d06 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:33.762524 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abb630-dbae-4d67-8476-c6ebf5413d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:33.859917 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-f5abb630-dbae-4d67-8476-c6ebf5413d06 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:45:33.864419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f5abb630-dbae-4d67-8476-c6ebf5413d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:45:34.016808 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 660/1308] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:45:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:45:34.292983 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392a425-3ff3-414c-824a-f288a12f2d6a None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:34.293571 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392a425-3ff3-414c-824a-f288a12f2d6a None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:34.293997 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392a425-3ff3-414c-824a-f288a12f2d6a None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:34.295794 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 649/1309] 10.222.0.59 () {54 vars in 965 bytes} [Fri Aug 26 00:45:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:45:34.302253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0c9e9-0928-464c-a126-5624c393a78f None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:34.302700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0c9e9-0928-464c-a126-5624c393a78f None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:34.303008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0c9e9-0928-464c-a126-5624c393a78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:34.396227 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-82b0c9e9-0928-464c-a126-5624c393a78f None None] MFA Rules not processed for user `4698a49ed4a04f58a488359a2742e5da`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:45:34.397594 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-82b0c9e9-0928-464c-a126-5624c393a78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:45:34.421961 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:45:34.486823 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 661/1310] 10.222.0.59 () {58 vars in 1062 bytes} [Fri Aug 26 00:45:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:45:34.497387 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392a425-3ff3-414c-824a-f288a12f2d6a None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:45:34.499303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-3392a425-3ff3-414c-824a-f288a12f2d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:45:34.642696 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392a425-3ff3-414c-824a-f288a12f2d6a None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:45:34.728721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e0d5ac-aa13-49f0-96c0-efe8746295aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:45:34.729342 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0d5ac-aa13-49f0-96c0-efe8746295aa service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:34.729627 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0d5ac-aa13-49f0-96c0-efe8746295aa service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:34.729901 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0d5ac-aa13-49f0-96c0-efe8746295aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:34.732316 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-81e0d5ac-aa13-49f0-96c0-efe8746295aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:45:34.794615 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:45:34.872331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e0d5ac-aa13-49f0-96c0-efe8746295aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:45:34.874212 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e0d5ac-aa13-49f0-96c0-efe8746295aa service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:45:34.895373 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 650/1311] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:45:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:45:37.190154 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df90694-e80a-48c0-b00f-40f1ea814c1a None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:37.190778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df90694-e80a-48c0-b00f-40f1ea814c1a None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:37.190778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df90694-e80a-48c0-b00f-40f1ea814c1a None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:37.192097 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 662/1312] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:45:37.197854 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38963c-deea-47e0-b24b-1e6b2266a652 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:37.198401 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38963c-deea-47e0-b24b-1e6b2266a652 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:37.198401 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38963c-deea-47e0-b24b-1e6b2266a652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:37.272134 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-ed38963c-deea-47e0-b24b-1e6b2266a652 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:45:37.273390 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-ed38963c-deea-47e0-b24b-1e6b2266a652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:45:37.369518 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 651/1313] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:45:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:45:37.651126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df90694-e80a-48c0-b00f-40f1ea814c1a None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:45:37.652754 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-9df90694-e80a-48c0-b00f-40f1ea814c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:45:37.755269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df90694-e80a-48c0-b00f-40f1ea814c1a None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:45:37.826323 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:45:37.828936 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1f947-dbe1-4675-9c80-bb0fc5000ba5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:45:37.829555 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1f947-dbe1-4675-9c80-bb0fc5000ba5 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:37.829838 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1f947-dbe1-4675-9c80-bb0fc5000ba5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:37.830113 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1f947-dbe1-4675-9c80-bb0fc5000ba5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:37.831736 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-1ba1f947-dbe1-4675-9c80-bb0fc5000ba5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:45:37.953702 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba1f947-dbe1-4675-9c80-bb0fc5000ba5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:45:37.955135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba1f947-dbe1-4675-9c80-bb0fc5000ba5 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:45:37.972966 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 663/1314] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:45:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:45:40.391710 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336af3a-be58-49bd-9a46-1d39dc93b9dc None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:40.392285 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336af3a-be58-49bd-9a46-1d39dc93b9dc None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:40.392285 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336af3a-be58-49bd-9a46-1d39dc93b9dc None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:40.393656 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 652/1315] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:45:40.398679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7c87b-9651-43d0-95cc-6273d7d5370c None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:40.399090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7c87b-9651-43d0-95cc-6273d7d5370c None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:40.399226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7c87b-9651-43d0-95cc-6273d7d5370c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:40.465638 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-f9e7c87b-9651-43d0-95cc-6273d7d5370c None None] MFA Rules not processed for user `b92a689fcc4842b3aa1d3a5c3f5e47b4`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:45:40.466840 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f9e7c87b-9651-43d0-95cc-6273d7d5370c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:45:40.518902 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:45:40.562400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 664/1316] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:45:40 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:45:40.841255 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336af3a-be58-49bd-9a46-1d39dc93b9dc None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:45:40.842824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-9336af3a-be58-49bd-9a46-1d39dc93b9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:45:40.852981 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:45:40.944612 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336af3a-be58-49bd-9a46-1d39dc93b9dc None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:45:41.015022 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d45a0-32fc-435c-983e-f4b1f1be0ddc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:45:41.015625 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d45a0-32fc-435c-983e-f4b1f1be0ddc service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:41.015901 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d45a0-32fc-435c-983e-f4b1f1be0ddc service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:41.016180 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d45a0-32fc-435c-983e-f4b1f1be0ddc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:41.017785 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-924d45a0-32fc-435c-983e-f4b1f1be0ddc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:45:41.078987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:45:41.125112 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d45a0-32fc-435c-983e-f4b1f1be0ddc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:45:41.126536 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d45a0-32fc-435c-983e-f4b1f1be0ddc service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:45:41.144206 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 653/1317] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:45:40 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:45:44.055210 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297743f-74c3-4a54-a32a-53eb5385b977 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:44.055759 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297743f-74c3-4a54-a32a-53eb5385b977 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:44.055855 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297743f-74c3-4a54-a32a-53eb5385b977 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:44.057197 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 665/1318] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:45:44.063338 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d3b28-912e-4db4-a4c7-f5fa1aa60a76 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:44.063827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d3b28-912e-4db4-a4c7-f5fa1aa60a76 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:44.063921 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d3b28-912e-4db4-a4c7-f5fa1aa60a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:44.129136 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-a45d3b28-912e-4db4-a4c7-f5fa1aa60a76 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:45:44.130364 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-a45d3b28-912e-4db4-a4c7-f5fa1aa60a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:45:44.226460 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 654/1319] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:45:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:45:44.521208 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681d20e-f42f-4119-a208-a6a0b1e95bee None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:44.521495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681d20e-f42f-4119-a208-a6a0b1e95bee None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:44.521773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681d20e-f42f-4119-a208-a6a0b1e95bee None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:44.523121 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 666/1320] 10.222.0.59 () {54 vars in 965 bytes} [Fri Aug 26 00:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:45:44.529646 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004975e-1992-4f7f-bd25-21a8cc871c17 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:44.529936 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004975e-1992-4f7f-bd25-21a8cc871c17 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:44.530211 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004975e-1992-4f7f-bd25-21a8cc871c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:44.552483 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:45:44.604306 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-2004975e-1992-4f7f-bd25-21a8cc871c17 None None] MFA Rules not processed for user `4698a49ed4a04f58a488359a2742e5da`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:45:44.606084 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-2004975e-1992-4f7f-bd25-21a8cc871c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:45:44.679124 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 655/1321] 10.222.0.59 () {58 vars in 1062 bytes} [Fri Aug 26 00:45:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:45:44.688419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681d20e-f42f-4119-a208-a6a0b1e95bee None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:45:44.690111 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-2681d20e-f42f-4119-a208-a6a0b1e95bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:45:44.794639 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681d20e-f42f-4119-a208-a6a0b1e95bee None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:45:44.866272 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a46e67-b424-435c-b3a7-b7b77d973f57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:45:44.866880 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a46e67-b424-435c-b3a7-b7b77d973f57 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:44.867160 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a46e67-b424-435c-b3a7-b7b77d973f57 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:44.867431 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a46e67-b424-435c-b3a7-b7b77d973f57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:44.869115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-b3a46e67-b424-435c-b3a7-b7b77d973f57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:45:44.876406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:45:44.976563 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a46e67-b424-435c-b3a7-b7b77d973f57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:45:44.977937 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a46e67-b424-435c-b3a7-b7b77d973f57 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:45:44.996916 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 667/1322] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:45:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:45:48.499098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc9e6d-6279-472b-991e-42f59c638528 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:48.499621 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc9e6d-6279-472b-991e-42f59c638528 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:48.499734 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc9e6d-6279-472b-991e-42f59c638528 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:48.501127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 656/1323] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:45:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:45:48.506590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6c15f-b5ca-4a0b-85d2-23e1c6c8ef5f None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:48.507002 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6c15f-b5ca-4a0b-85d2-23e1c6c8ef5f None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:48.507150 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6c15f-b5ca-4a0b-85d2-23e1c6c8ef5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:48.573068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-e8d6c15f-b5ca-4a0b-85d2-23e1c6c8ef5f None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:45:48.574269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-e8d6c15f-b5ca-4a0b-85d2-23e1c6c8ef5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:45:48.634160 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:45:48.672812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 668/1324] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:45:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:45:48.963797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc9e6d-6279-472b-991e-42f59c638528 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:45:48.966104 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-93bc9e6d-6279-472b-991e-42f59c638528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:45:49.080585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc9e6d-6279-472b-991e-42f59c638528 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:45:49.094916 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:45:49.157822 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304c6af-77f2-4323-a7d9-9b4e953e5141 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:45:49.158432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304c6af-77f2-4323-a7d9-9b4e953e5141 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:49.158740 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304c6af-77f2-4323-a7d9-9b4e953e5141 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:49.159010 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304c6af-77f2-4323-a7d9-9b4e953e5141 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:49.160647 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-7304c6af-77f2-4323-a7d9-9b4e953e5141 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:45:49.268957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7304c6af-77f2-4323-a7d9-9b4e953e5141 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:45:49.270315 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7304c6af-77f2-4323-a7d9-9b4e953e5141 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:45:49.289395 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 657/1325] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:45:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:45:51.261000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56375782-6553-45ce-96bb-45109b0f7def None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:51.262080 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56375782-6553-45ce-96bb-45109b0f7def None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:51.262194 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56375782-6553-45ce-96bb-45109b0f7def None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:51.263509 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 669/1326] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:45:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:45:51.268770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df9cae-1bc4-4158-95dd-2df88ae669ac None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:51.269227 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df9cae-1bc4-4158-95dd-2df88ae669ac None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:51.269338 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df9cae-1bc4-4158-95dd-2df88ae669ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:51.318417 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:45:51.336741 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-22df9cae-1bc4-4158-95dd-2df88ae669ac None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:45:51.337954 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-22df9cae-1bc4-4158-95dd-2df88ae669ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:45:51.437755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 658/1327] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:45:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:45:51.730397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56375782-6553-45ce-96bb-45109b0f7def None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:45:51.731954 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-56375782-6553-45ce-96bb-45109b0f7def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:45:51.844385 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56375782-6553-45ce-96bb-45109b0f7def None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:45:51.919517 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf85252-5109-46f2-9285-56eb390945e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:45:51.920139 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf85252-5109-46f2-9285-56eb390945e9 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:51.920420 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf85252-5109-46f2-9285-56eb390945e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:51.920725 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf85252-5109-46f2-9285-56eb390945e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:51.922358 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-ecf85252-5109-46f2-9285-56eb390945e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:45:51.939289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:45:52.042350 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf85252-5109-46f2-9285-56eb390945e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:45:52.045252 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf85252-5109-46f2-9285-56eb390945e9 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:45:52.064216 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 670/1328] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:45:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:45:54.036057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b0be6-16f8-4c05-b1e7-d6edd011dc2f None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:54.036654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b0be6-16f8-4c05-b1e7-d6edd011dc2f None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:54.036654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b0be6-16f8-4c05-b1e7-d6edd011dc2f None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:54.038027 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 659/1329] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:45:54.043566 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661cb80-40be-4c7c-be0f-5000fe0b8bab None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:54.043994 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661cb80-40be-4c7c-be0f-5000fe0b8bab None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:54.044146 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661cb80-40be-4c7c-be0f-5000fe0b8bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:54.112166 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-2661cb80-40be-4c7c-be0f-5000fe0b8bab None None] MFA Rules not processed for user `b92a689fcc4842b3aa1d3a5c3f5e47b4`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:45:54.113521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-2661cb80-40be-4c7c-be0f-5000fe0b8bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:45:54.182314 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:45:54.217160 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 671/1330] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:45:54 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:45:54.502546 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b0be6-16f8-4c05-b1e7-d6edd011dc2f None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:45:54.504095 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-890b0be6-16f8-4c05-b1e7-d6edd011dc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:45:54.622404 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b0be6-16f8-4c05-b1e7-d6edd011dc2f None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:45:54.642520 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:45:54.698497 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0c19e-5060-4ee1-8157-04a95646d03d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:45:54.699631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0c19e-5060-4ee1-8157-04a95646d03d service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:54.699911 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0c19e-5060-4ee1-8157-04a95646d03d service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:54.700184 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0c19e-5060-4ee1-8157-04a95646d03d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:54.701840 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-9cd0c19e-5060-4ee1-8157-04a95646d03d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:45:54.811355 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0c19e-5060-4ee1-8157-04a95646d03d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:45:54.812761 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0c19e-5060-4ee1-8157-04a95646d03d service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:45:54.831674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 660/1331] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:45:54 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:45:57.193530 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2b4b4-03cf-4bf8-b1ae-ee64c471f7ab None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:57.194137 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2b4b4-03cf-4bf8-b1ae-ee64c471f7ab None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:57.194137 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2b4b4-03cf-4bf8-b1ae-ee64c471f7ab None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:57.195423 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 672/1332] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:45:57.201188 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4d3f1-3d01-42ca-b044-8934be88fe1a None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:57.201618 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4d3f1-3d01-42ca-b044-8934be88fe1a None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:57.201734 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4d3f1-3d01-42ca-b044-8934be88fe1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:57.263573 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:45:57.266331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-19c4d3f1-3d01-42ca-b044-8934be88fe1a None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:45:57.267519 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-19c4d3f1-3d01-42ca-b044-8934be88fe1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:45:57.361675 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 661/1333] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:45:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:45:57.646101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d2b4b4-03cf-4bf8-b1ae-ee64c471f7ab None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:45:57.647696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a5d2b4b4-03cf-4bf8-b1ae-ee64c471f7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:45:57.754009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d2b4b4-03cf-4bf8-b1ae-ee64c471f7ab None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:45:57.828732 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a597a-e42b-4de9-89aa-978823101d74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:45:57.829338 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a597a-e42b-4de9-89aa-978823101d74 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:45:57.829612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a597a-e42b-4de9-89aa-978823101d74 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:45:57.829880 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a597a-e42b-4de9-89aa-978823101d74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:45:57.831465 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-324a597a-e42b-4de9-89aa-978823101d74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:45:57.974500 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:45:58.056701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a597a-e42b-4de9-89aa-978823101d74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:45:58.058072 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a597a-e42b-4de9-89aa-978823101d74 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:45:58.075778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 673/1334] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:45:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:46:00.002520 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff3d2b-d247-4712-b083-82079d2ef78c None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:00.003064 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff3d2b-d247-4712-b083-82079d2ef78c None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:00.003064 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff3d2b-d247-4712-b083-82079d2ef78c None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:00.004394 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 662/1335] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:46:00.009483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef0323-fc49-4685-8e98-c7b857ecc34f None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:00.009902 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef0323-fc49-4685-8e98-c7b857ecc34f None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:00.010035 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef0323-fc49-4685-8e98-c7b857ecc34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:00.080113 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-1eef0323-fc49-4685-8e98-c7b857ecc34f None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:46:00.082255 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-1eef0323-fc49-4685-8e98-c7b857ecc34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:00.184206 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:00.206028 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 674/1336] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:46:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:46:00.502064 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff3d2b-d247-4712-b083-82079d2ef78c None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:46:00.503607 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-5bff3d2b-d247-4712-b083-82079d2ef78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:00.608364 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff3d2b-d247-4712-b083-82079d2ef78c None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:46:00.636403 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:00.680270 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d990e-bdd8-4170-a91f-5210833baf5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:46:00.680908 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d990e-bdd8-4170-a91f-5210833baf5a service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:00.681187 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d990e-bdd8-4170-a91f-5210833baf5a service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:00.681537 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d990e-bdd8-4170-a91f-5210833baf5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:00.683715 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-cb8d990e-bdd8-4170-a91f-5210833baf5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:00.787839 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d990e-bdd8-4170-a91f-5210833baf5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:46:00.789232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d990e-bdd8-4170-a91f-5210833baf5a service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:46:00.807789 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 663/1337] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:46:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:46:03.416421 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede254d-0525-4842-8146-e05b60c34f83 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:03.417056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede254d-0525-4842-8146-e05b60c34f83 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:03.417056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede254d-0525-4842-8146-e05b60c34f83 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:03.418935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 675/1338] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:46:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:46:03.423904 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd06c35-e325-48a8-ae1e-c6ae1fc296e2 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:03.424284 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd06c35-e325-48a8-ae1e-c6ae1fc296e2 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:03.424456 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd06c35-e325-48a8-ae1e-c6ae1fc296e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:03.491114 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-fcd06c35-e325-48a8-ae1e-c6ae1fc296e2 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:46:03.492350 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-fcd06c35-e325-48a8-ae1e-c6ae1fc296e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:03.500898 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:03.586211 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 664/1339] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:46:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:46:03.864056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede254d-0525-4842-8146-e05b60c34f83 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:46:03.865627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-4ede254d-0525-4842-8146-e05b60c34f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:03.969362 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede254d-0525-4842-8146-e05b60c34f83 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:46:04.042117 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa304233-6a0b-42f6-917c-0ebd07ab03aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:46:04.042725 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa304233-6a0b-42f6-917c-0ebd07ab03aa service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:04.043006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa304233-6a0b-42f6-917c-0ebd07ab03aa service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:04.043278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa304233-6a0b-42f6-917c-0ebd07ab03aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:04.044910 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-aa304233-6a0b-42f6-917c-0ebd07ab03aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:04.079790 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:04.150394 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa304233-6a0b-42f6-917c-0ebd07ab03aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:46:04.152651 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa304233-6a0b-42f6-917c-0ebd07ab03aa service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:46:04.171335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 676/1340] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:46:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:46:08.455391 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4db343-1214-4bb1-9e73-68d3a591de8d None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:08.455962 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4db343-1214-4bb1-9e73-68d3a591de8d None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:08.455962 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4db343-1214-4bb1-9e73-68d3a591de8d None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:08.457329 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 665/1341] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:46:08.462609 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7987d-25c2-4a0b-8f27-43f0a5b8b91a None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:08.463076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7987d-25c2-4a0b-8f27-43f0a5b8b91a None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:08.463181 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7987d-25c2-4a0b-8f27-43f0a5b8b91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:08.529314 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-09b7987d-25c2-4a0b-8f27-43f0a5b8b91a None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:46:08.530536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-09b7987d-25c2-4a0b-8f27-43f0a5b8b91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:08.622056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:08.633589 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 677/1342] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:46:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:46:08.915233 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4db343-1214-4bb1-9e73-68d3a591de8d None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:46:08.917444 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-4a4db343-1214-4bb1-9e73-68d3a591de8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:09.020220 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4db343-1214-4bb1-9e73-68d3a591de8d None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:46:09.060746 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:09.098830 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ecc72-d4bd-49ed-bb75-91205206c568 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:46:09.105114 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ecc72-d4bd-49ed-bb75-91205206c568 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:09.105114 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ecc72-d4bd-49ed-bb75-91205206c568 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:09.105114 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ecc72-d4bd-49ed-bb75-91205206c568 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:09.105114 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-055ecc72-d4bd-49ed-bb75-91205206c568 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:09.211778 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055ecc72-d4bd-49ed-bb75-91205206c568 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:46:09.213172 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055ecc72-d4bd-49ed-bb75-91205206c568 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:46:09.233039 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 666/1343] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:46:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:46:12.212345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4703b-798b-4c78-9d20-0b1f55c7d2db None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:12.212919 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4703b-798b-4c78-9d20-0b1f55c7d2db None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:12.212919 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4703b-798b-4c78-9d20-0b1f55c7d2db None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:12.214660 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 678/1344] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:46:12.219956 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b1504-aa87-4bc6-953e-33e75cc866b2 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:12.220374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b1504-aa87-4bc6-953e-33e75cc866b2 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:12.220653 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b1504-aa87-4bc6-953e-33e75cc866b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:12.288228 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-350b1504-aa87-4bc6-953e-33e75cc866b2 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:46:12.289488 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-350b1504-aa87-4bc6-953e-33e75cc866b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:12.310331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:12.387274 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 667/1345] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:46:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:46:12.673846 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4703b-798b-4c78-9d20-0b1f55c7d2db None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:46:12.675500 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-1cd4703b-798b-4c78-9d20-0b1f55c7d2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:12.778006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4703b-798b-4c78-9d20-0b1f55c7d2db None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:46:12.858361 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d5b79-5b2b-4415-962f-2bda8356ab53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:46:12.858943 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d5b79-5b2b-4415-962f-2bda8356ab53 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:12.859219 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d5b79-5b2b-4415-962f-2bda8356ab53 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:12.859487 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d5b79-5b2b-4415-962f-2bda8356ab53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:12.861126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-c80d5b79-5b2b-4415-962f-2bda8356ab53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:12.907901 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:12.975920 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d5b79-5b2b-4415-962f-2bda8356ab53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:46:12.977388 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d5b79-5b2b-4415-962f-2bda8356ab53 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:46:12.996456 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 679/1346] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:46:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:46:17.287218 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04243b2-101f-4384-bac7-b729ed0b46c8 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:17.287730 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04243b2-101f-4384-bac7-b729ed0b46c8 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:17.287844 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04243b2-101f-4384-bac7-b729ed0b46c8 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:17.289151 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 668/1347] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:46:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:46:17.295097 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71543c9-b6a7-46da-9954-125888a3c421 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:17.295563 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71543c9-b6a7-46da-9954-125888a3c421 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:17.295667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71543c9-b6a7-46da-9954-125888a3c421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:17.367755 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-d71543c9-b6a7-46da-9954-125888a3c421 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:46:17.369001 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-d71543c9-b6a7-46da-9954-125888a3c421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:17.476685 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:17.479935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 680/1348] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:46:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:46:17.763440 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04243b2-101f-4384-bac7-b729ed0b46c8 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:46:17.765115 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-b04243b2-101f-4384-bac7-b729ed0b46c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:17.875395 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04243b2-101f-4384-bac7-b729ed0b46c8 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:46:17.923017 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:17.948891 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd028217-b5c2-4577-999b-dcd9979dc286 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:46:17.949512 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd028217-b5c2-4577-999b-dcd9979dc286 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:17.949824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd028217-b5c2-4577-999b-dcd9979dc286 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:17.950097 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd028217-b5c2-4577-999b-dcd9979dc286 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:17.951740 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-fd028217-b5c2-4577-999b-dcd9979dc286 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:18.057728 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd028217-b5c2-4577-999b-dcd9979dc286 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:46:18.059083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd028217-b5c2-4577-999b-dcd9979dc286 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:46:18.077796 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 669/1349] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:46:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:46:22.964844 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f874d3-b093-420a-8ff4-9d38f1dfd633 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:22.964844 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f874d3-b093-420a-8ff4-9d38f1dfd633 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:22.965459 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f874d3-b093-420a-8ff4-9d38f1dfd633 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:22.966740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 681/1350] 10.222.0.59 () {54 vars in 921 bytes} [Fri Aug 26 00:46:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:46:22.972199 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a6c53-2309-41ff-8d0f-ac7d4ef84cf7 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:22.973055 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a6c53-2309-41ff-8d0f-ac7d4ef84cf7 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:22.973618 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a6c53-2309-41ff-8d0f-ac7d4ef84cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:23.062969 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-3f0a6c53-2309-41ff-8d0f-ac7d4ef84cf7 None None] MFA Rules not processed for user `2876f4182f0949039b862f053533fe85`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:46:23.064222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-3f0a6c53-2309-41ff-8d0f-ac7d4ef84cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:23.108041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:23.262267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 670/1351] 10.222.0.59 () {58 vars in 1018 bytes} [Fri Aug 26 00:46:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:46:27.344311 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097bd56d-793f-40ba-96f3-f70f4cee925a None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:27.344875 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097bd56d-793f-40ba-96f3-f70f4cee925a None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:27.344875 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097bd56d-793f-40ba-96f3-f70f4cee925a None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:27.346219 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 682/1352] 10.222.0.59 () {54 vars in 921 bytes} [Fri Aug 26 00:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:46:27.351970 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb39a70-bf71-4069-9aa9-c2e7c964ce03 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:27.352401 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb39a70-bf71-4069-9aa9-c2e7c964ce03 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:27.352719 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb39a70-bf71-4069-9aa9-c2e7c964ce03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:27.438053 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-4cb39a70-bf71-4069-9aa9-c2e7c964ce03 None None] MFA Rules not processed for user `2876f4182f0949039b862f053533fe85`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:46:27.439166 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-4cb39a70-bf71-4069-9aa9-c2e7c964ce03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:27.530296 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:27.566692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 671/1353] 10.222.0.59 () {58 vars in 1018 bytes} [Fri Aug 26 00:46:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:46:29.033667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9885a9d-c615-48b6-b10f-f279d805f279 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:29.034231 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9885a9d-c615-48b6-b10f-f279d805f279 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:29.034231 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9885a9d-c615-48b6-b10f-f279d805f279 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:29.035604 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 683/1354] 10.222.0.59 () {54 vars in 921 bytes} [Fri Aug 26 00:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:46:29.040872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7e920-38c9-4323-a18a-f8dae55902bb None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:29.041305 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7e920-38c9-4323-a18a-f8dae55902bb None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:29.041413 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7e920-38c9-4323-a18a-f8dae55902bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:29.117970 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-d9a7e920-38c9-4323-a18a-f8dae55902bb None None] MFA Rules not processed for user `2876f4182f0949039b862f053533fe85`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:46:29.119223 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-d9a7e920-38c9-4323-a18a-f8dae55902bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:29.229925 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 672/1355] 10.222.0.59 () {58 vars in 1018 bytes} [Fri Aug 26 00:46:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:46:32.497444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc790c1-d5a5-4b60-8e9a-6c7bd20bb219 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:32.498021 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc790c1-d5a5-4b60-8e9a-6c7bd20bb219 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:32.498129 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc790c1-d5a5-4b60-8e9a-6c7bd20bb219 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:32.499391 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 684/1356] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:46:32.504714 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86084eae-afa3-40eb-9f9a-9ffd2a6a53a4 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:32.505227 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86084eae-afa3-40eb-9f9a-9ffd2a6a53a4 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:32.505227 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86084eae-afa3-40eb-9f9a-9ffd2a6a53a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:32.542162 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:32.585803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-86084eae-afa3-40eb-9f9a-9ffd2a6a53a4 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:46:32.587000 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-86084eae-afa3-40eb-9f9a-9ffd2a6a53a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:32.715911 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 673/1357] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:46:33.017528 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e653a91-87f3-46fe-a8e9-c0a66ac4d54e None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:33.017807 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e653a91-87f3-46fe-a8e9-c0a66ac4d54e None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:33.018296 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e653a91-87f3-46fe-a8e9-c0a66ac4d54e None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:33.019630 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 685/1358] 10.222.0.59 () {54 vars in 962 bytes} [Fri Aug 26 00:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:46:33.030297 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c5999-bc73-4268-8c86-332cb8bbdd90 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:33.030757 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c5999-bc73-4268-8c86-332cb8bbdd90 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:33.031174 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c5999-bc73-4268-8c86-332cb8bbdd90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:33.117092 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-282c5999-bc73-4268-8c86-332cb8bbdd90 None None] MFA Rules not processed for user `0602593263ca4b05a657d5b7697f932f`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:46:33.118429 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-282c5999-bc73-4268-8c86-332cb8bbdd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:33.172398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:33.251901 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 674/1359] 10.222.0.59 () {58 vars in 1059 bytes} [Fri Aug 26 00:46:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:46:33.259886 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e653a91-87f3-46fe-a8e9-c0a66ac4d54e None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:46:33.261653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-6e653a91-87f3-46fe-a8e9-c0a66ac4d54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:33.416984 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e653a91-87f3-46fe-a8e9-c0a66ac4d54e None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:46:33.502461 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:33.547537 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c7c82-33e8-4c52-bcf2-f5d25e77e547 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:46:33.548256 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c7c82-33e8-4c52-bcf2-f5d25e77e547 service nova] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:33.548711 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c7c82-33e8-4c52-bcf2-f5d25e77e547 service nova] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:33.549055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c7c82-33e8-4c52-bcf2-f5d25e77e547 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:33.550820 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-9b8c7c82-33e8-4c52-bcf2-f5d25e77e547 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:33.643551 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c7c82-33e8-4c52-bcf2-f5d25e77e547 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:46:33.645177 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c7c82-33e8-4c52-bcf2-f5d25e77e547 service nova] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:46:33.664130 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 686/1360] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:46:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:46:35.824119 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd0275-d7a0-4ced-a728-73a9cd38527b None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:35.824593 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd0275-d7a0-4ced-a728-73a9cd38527b None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:35.824682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd0275-d7a0-4ced-a728-73a9cd38527b None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:35.826036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 675/1361] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:46:35.831702 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433aa0a5-eb55-4aec-99d8-1064c9fab368 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:35.832384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433aa0a5-eb55-4aec-99d8-1064c9fab368 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:35.832874 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433aa0a5-eb55-4aec-99d8-1064c9fab368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:35.903902 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-433aa0a5-eb55-4aec-99d8-1064c9fab368 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:46:35.905220 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-433aa0a5-eb55-4aec-99d8-1064c9fab368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:35.945380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:36.006492 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 687/1362] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:46:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:46:36.300247 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd0275-d7a0-4ced-a728-73a9cd38527b None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:46:36.302060 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-55fd0275-d7a0-4ced-a728-73a9cd38527b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:36.488121 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd0275-d7a0-4ced-a728-73a9cd38527b None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:46:36.498761 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:36.606413 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41789642-9f29-4bd2-8332-2e1397d7a6f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:46:36.607022 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41789642-9f29-4bd2-8332-2e1397d7a6f8 service nova] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:36.607294 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41789642-9f29-4bd2-8332-2e1397d7a6f8 service nova] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:36.607558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41789642-9f29-4bd2-8332-2e1397d7a6f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:36.609184 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-41789642-9f29-4bd2-8332-2e1397d7a6f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:36.702083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41789642-9f29-4bd2-8332-2e1397d7a6f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:46:36.703693 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41789642-9f29-4bd2-8332-2e1397d7a6f8 service nova] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:46:36.723965 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 676/1363] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:46:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:46:39.164948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b64d1aa-cdb1-4994-b70f-003550ab2e7b None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:39.166323 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b64d1aa-cdb1-4994-b70f-003550ab2e7b None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:39.166612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b64d1aa-cdb1-4994-b70f-003550ab2e7b None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:39.168621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 688/1364] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:46:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:46:39.174267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52298675-cb40-4123-8d3b-14b4e35d2046 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:39.175076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52298675-cb40-4123-8d3b-14b4e35d2046 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:39.175508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52298675-cb40-4123-8d3b-14b4e35d2046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:39.191612 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:39.249141 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-52298675-cb40-4123-8d3b-14b4e35d2046 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:46:39.250504 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-52298675-cb40-4123-8d3b-14b4e35d2046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:39.352488 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 677/1365] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:46:39.647993 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7614455-10a2-4c25-a4e1-67fd4bb58366 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:39.648274 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7614455-10a2-4c25-a4e1-67fd4bb58366 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:39.648749 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7614455-10a2-4c25-a4e1-67fd4bb58366 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:39.650234 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 689/1366] 10.222.0.59 () {54 vars in 962 bytes} [Fri Aug 26 00:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:46:39.657458 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b0a52-91ad-4bec-8c85-274d5412195f None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:39.657813 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b0a52-91ad-4bec-8c85-274d5412195f None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:39.658159 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b0a52-91ad-4bec-8c85-274d5412195f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:39.740730 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-b07b0a52-91ad-4bec-8c85-274d5412195f None None] MFA Rules not processed for user `0602593263ca4b05a657d5b7697f932f`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:46:39.741960 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-b07b0a52-91ad-4bec-8c85-274d5412195f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:39.778680 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:39.866921 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 678/1367] 10.222.0.59 () {58 vars in 1059 bytes} [Fri Aug 26 00:46:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:46:39.875120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7614455-10a2-4c25-a4e1-67fd4bb58366 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:46:39.876883 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-b7614455-10a2-4c25-a4e1-67fd4bb58366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:40.045499 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7614455-10a2-4c25-a4e1-67fd4bb58366 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:46:40.067401 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:40.173557 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92436baf-9a5f-4839-9ae6-a41f3e4fbe75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:46:40.174363 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92436baf-9a5f-4839-9ae6-a41f3e4fbe75 service nova] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:40.174479 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92436baf-9a5f-4839-9ae6-a41f3e4fbe75 service nova] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:40.174723 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92436baf-9a5f-4839-9ae6-a41f3e4fbe75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:40.176724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-92436baf-9a5f-4839-9ae6-a41f3e4fbe75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:40.270909 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92436baf-9a5f-4839-9ae6-a41f3e4fbe75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:46:40.272694 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92436baf-9a5f-4839-9ae6-a41f3e4fbe75 service nova] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:46:40.292816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 690/1368] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:46:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:46:40.940689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acbac7-6f96-4fe0-b467-63709c442ec6 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:40.941277 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acbac7-6f96-4fe0-b467-63709c442ec6 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:40.941277 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acbac7-6f96-4fe0-b467-63709c442ec6 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:40.942723 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 679/1369] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:46:40.947860 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cdab9-d22b-4ce3-bce3-4bb050212d20 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:40.948731 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cdab9-d22b-4ce3-bce3-4bb050212d20 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:40.949006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cdab9-d22b-4ce3-bce3-4bb050212d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:40.984913 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:41.023373 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-180cdab9-d22b-4ce3-bce3-4bb050212d20 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:46:41.024623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-180cdab9-d22b-4ce3-bce3-4bb050212d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:41.131426 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 691/1370] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:46:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:46:41.733739 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8303f-0c4a-4446-bcb4-c70bac5d49ab None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:41.733739 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8303f-0c4a-4446-bcb4-c70bac5d49ab None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:41.733739 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8303f-0c4a-4446-bcb4-c70bac5d49ab None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:41.733739 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 680/1371] 10.222.0.59 () {54 vars in 964 bytes} [Fri Aug 26 00:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:46:41.738681 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ecd86-0b2a-4a27-87f9-0b971d840a49 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:41.739221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ecd86-0b2a-4a27-87f9-0b971d840a49 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:41.739221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ecd86-0b2a-4a27-87f9-0b971d840a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:41.828056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-c56ecd86-0b2a-4a27-87f9-0b971d840a49 None None] MFA Rules not processed for user `c88202614775418ba94f478f03431a9b`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:46:41.829318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-c56ecd86-0b2a-4a27-87f9-0b971d840a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:41.880114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:41.912516 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 692/1372] 10.222.0.59 () {58 vars in 1061 bytes} [Fri Aug 26 00:46:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:46:41.928912 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8303f-0c4a-4446-bcb4-c70bac5d49ab None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:46:41.930465 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-36b8303f-0c4a-4446-bcb4-c70bac5d49ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:42.041447 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8303f-0c4a-4446-bcb4-c70bac5d49ab None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:46:42.088337 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:42.122793 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92272c-cb77-43b4-81b7-afeb49348a3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:46:42.123408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92272c-cb77-43b4-81b7-afeb49348a3e service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:42.123691 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92272c-cb77-43b4-81b7-afeb49348a3e service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:42.123991 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92272c-cb77-43b4-81b7-afeb49348a3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:42.125705 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-6f92272c-cb77-43b4-81b7-afeb49348a3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:42.245430 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92272c-cb77-43b4-81b7-afeb49348a3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:46:42.247218 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92272c-cb77-43b4-81b7-afeb49348a3e service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:46:42.267468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 681/1373] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:46:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:46:42.442595 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2cd8f-02d0-4d27-ad36-a107da7b53e4 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:42.442595 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2cd8f-02d0-4d27-ad36-a107da7b53e4 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:42.442595 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2cd8f-02d0-4d27-ad36-a107da7b53e4 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:42.447734 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 693/1374] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:46:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:46:42.448580 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ba431-6051-4b08-ac5e-48625b6075be None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:42.448580 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ba431-6051-4b08-ac5e-48625b6075be None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:42.448941 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ba431-6051-4b08-ac5e-48625b6075be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:42.516671 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-051ba431-6051-4b08-ac5e-48625b6075be None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:46:42.517908 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-051ba431-6051-4b08-ac5e-48625b6075be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:42.543813 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:42.619913 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 682/1375] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:46:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:46:42.906417 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2cd8f-02d0-4d27-ad36-a107da7b53e4 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:46:42.908287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a6c2cd8f-02d0-4d27-ad36-a107da7b53e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:43.055592 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2cd8f-02d0-4d27-ad36-a107da7b53e4 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:46:43.108397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:43.170255 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd78fbf-be98-419e-aead-fb41db162328 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:46:43.170787 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd78fbf-be98-419e-aead-fb41db162328 service nova] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:43.171067 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd78fbf-be98-419e-aead-fb41db162328 service nova] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:43.171339 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd78fbf-be98-419e-aead-fb41db162328 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:43.173851 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-ecd78fbf-be98-419e-aead-fb41db162328 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:43.270689 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd78fbf-be98-419e-aead-fb41db162328 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:46:43.272304 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd78fbf-be98-419e-aead-fb41db162328 service nova] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:46:43.291375 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 694/1376] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:46:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:46:45.332292 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aafd54-98c2-4e20-913c-e7fb06f3079d None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:45.332865 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aafd54-98c2-4e20-913c-e7fb06f3079d None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:45.332865 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aafd54-98c2-4e20-913c-e7fb06f3079d None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:45.334218 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 683/1377] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:46:45.339496 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296a9c5-2aa0-49ea-aba5-f831f5cdf702 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:45.339922 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296a9c5-2aa0-49ea-aba5-f831f5cdf702 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:45.340181 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296a9c5-2aa0-49ea-aba5-f831f5cdf702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:45.408047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-1296a9c5-2aa0-49ea-aba5-f831f5cdf702 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:46:45.409287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-1296a9c5-2aa0-49ea-aba5-f831f5cdf702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:45.416165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:45.526492 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 695/1378] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:46:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:46:45.824942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aafd54-98c2-4e20-913c-e7fb06f3079d None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:46:45.826494 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-f5aafd54-98c2-4e20-913c-e7fb06f3079d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:46.004872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aafd54-98c2-4e20-913c-e7fb06f3079d None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:46:46.008831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:46.137104 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8aca4d-e69d-4be2-b660-16de830890e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:46:46.137894 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8aca4d-e69d-4be2-b660-16de830890e8 service nova] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:46.138273 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8aca4d-e69d-4be2-b660-16de830890e8 service nova] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:46.138642 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8aca4d-e69d-4be2-b660-16de830890e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:46.140364 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-be8aca4d-e69d-4be2-b660-16de830890e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:46.233850 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8aca4d-e69d-4be2-b660-16de830890e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:46:46.235577 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8aca4d-e69d-4be2-b660-16de830890e8 service nova] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:46:46.257289 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 684/1379] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:46:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:46:48.355877 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1bfbc-44af-4973-a5d8-4a451d99884a None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:48.356444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1bfbc-44af-4973-a5d8-4a451d99884a None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:48.356444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1bfbc-44af-4973-a5d8-4a451d99884a None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:48.357946 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 696/1380] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:46:48.363179 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df51e6e-21e8-4dbc-b6f2-4c8f77f5cf5b None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:48.363590 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df51e6e-21e8-4dbc-b6f2-4c8f77f5cf5b None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:48.363966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df51e6e-21e8-4dbc-b6f2-4c8f77f5cf5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:48.378519 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:48.443575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-1df51e6e-21e8-4dbc-b6f2-4c8f77f5cf5b None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:46:48.444793 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-1df51e6e-21e8-4dbc-b6f2-4c8f77f5cf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:48.552608 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 685/1381] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:46:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:46:48.854309 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1bfbc-44af-4973-a5d8-4a451d99884a None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:46:48.856328 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-28e1bfbc-44af-4973-a5d8-4a451d99884a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:49.019584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:49.030595 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1bfbc-44af-4973-a5d8-4a451d99884a None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:46:49.149673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c5f83-c8f4-41f7-8463-d112e77c14f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:46:49.150358 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c5f83-c8f4-41f7-8463-d112e77c14f2 service nova] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:49.150741 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c5f83-c8f4-41f7-8463-d112e77c14f2 service nova] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:49.151131 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c5f83-c8f4-41f7-8463-d112e77c14f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:49.152879 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-d31c5f83-c8f4-41f7-8463-d112e77c14f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:49.245329 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c5f83-c8f4-41f7-8463-d112e77c14f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:46:49.247080 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c5f83-c8f4-41f7-8463-d112e77c14f2 service nova] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:46:49.258078 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:49.266609 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 697/1382] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:46:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:46:51.759927 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731790e7-52eb-4300-b7d0-643edbe74847 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:51.761011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731790e7-52eb-4300-b7d0-643edbe74847 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:51.761011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731790e7-52eb-4300-b7d0-643edbe74847 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:51.762273 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 686/1383] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:46:51.768912 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974df85-47ff-4288-9f91-19a9ea9504d7 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:51.769393 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974df85-47ff-4288-9f91-19a9ea9504d7 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:51.769494 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974df85-47ff-4288-9f91-19a9ea9504d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:51.840741 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-d974df85-47ff-4288-9f91-19a9ea9504d7 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:46:51.841964 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-d974df85-47ff-4288-9f91-19a9ea9504d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:51.952998 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 698/1384] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:46:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:46:52.240885 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731790e7-52eb-4300-b7d0-643edbe74847 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:46:52.242494 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-731790e7-52eb-4300-b7d0-643edbe74847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:52.462390 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:52.530769 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731790e7-52eb-4300-b7d0-643edbe74847 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:46:52.659234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1c044-2963-4707-83a1-27d3a0099b25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:46:52.659849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1c044-2963-4707-83a1-27d3a0099b25 service nova] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:52.660133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1c044-2963-4707-83a1-27d3a0099b25 service nova] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:52.660406 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1c044-2963-4707-83a1-27d3a0099b25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:52.662039 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-3bd1c044-2963-4707-83a1-27d3a0099b25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:52.712173 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:52.766921 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1c044-2963-4707-83a1-27d3a0099b25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:46:52.769759 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1c044-2963-4707-83a1-27d3a0099b25 service nova] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:46:52.801398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 687/1385] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:46:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:46:54.895689 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae3ab6-b91a-4da2-91c6-75cc73e05178 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:54.896368 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae3ab6-b91a-4da2-91c6-75cc73e05178 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:54.896368 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae3ab6-b91a-4da2-91c6-75cc73e05178 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:54.897709 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 699/1386] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:46:54.903985 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800da52-544a-49c8-b2b1-9b309e3588bd None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:54.904448 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800da52-544a-49c8-b2b1-9b309e3588bd None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:54.904598 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800da52-544a-49c8-b2b1-9b309e3588bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:54.972186 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-e800da52-544a-49c8-b2b1-9b309e3588bd None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:46:54.973351 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-e800da52-544a-49c8-b2b1-9b309e3588bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:55.086217 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 688/1387] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:46:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:46:55.392996 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae3ab6-b91a-4da2-91c6-75cc73e05178 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:46:55.394601 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-31ae3ab6-b91a-4da2-91c6-75cc73e05178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:55.485857 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:55.571320 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae3ab6-b91a-4da2-91c6-75cc73e05178 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:46:55.697015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd69671-7e3f-4c01-87cd-473cff949cbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:46:55.697526 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd69671-7e3f-4c01-87cd-473cff949cbb service nova] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:55.697806 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd69671-7e3f-4c01-87cd-473cff949cbb service nova] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:55.698078 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd69671-7e3f-4c01-87cd-473cff949cbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:55.699689 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-2fd69671-7e3f-4c01-87cd-473cff949cbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:55.739612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:55.795283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd69671-7e3f-4c01-87cd-473cff949cbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:46:55.797030 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd69671-7e3f-4c01-87cd-473cff949cbb service nova] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:46:55.815813 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 700/1388] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:46:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:46:57.761074 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9c803-ad3c-486c-9823-8e380b70137d None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:57.761658 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9c803-ad3c-486c-9823-8e380b70137d None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:57.761658 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9c803-ad3c-486c-9823-8e380b70137d None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:57.762867 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 689/1389] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:46:57.768152 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f647ae-558d-4cdc-897f-802a1c126e71 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:57.768639 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f647ae-558d-4cdc-897f-802a1c126e71 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:57.768753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f647ae-558d-4cdc-897f-802a1c126e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:57.837804 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-77f647ae-558d-4cdc-897f-802a1c126e71 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:46:57.839058 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-77f647ae-558d-4cdc-897f-802a1c126e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:57.900241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd09444-449d-4c9c-b1f2-8b86a616aeaf None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:57.900590 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd09444-449d-4c9c-b1f2-8b86a616aeaf None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:57.900851 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd09444-449d-4c9c-b1f2-8b86a616aeaf None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:57.902329 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 690/1390] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:46:57.907279 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4143e2-d0e4-49dc-ba95-35ce1a74f77b None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:57.907553 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4143e2-d0e4-49dc-ba95-35ce1a74f77b None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:57.907930 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4143e2-d0e4-49dc-ba95-35ce1a74f77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:57.940084 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 701/1391] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:46:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:46:57.943563 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:57.974794 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-ba4143e2-d0e4-49dc-ba95-35ce1a74f77b None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:46:57.975987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-ba4143e2-d0e4-49dc-ba95-35ce1a74f77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:58.072757 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 691/1392] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:46:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:46:58.244635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048de07f-ecee-45c1-bcba-d666548a767a None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:58.245468 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048de07f-ecee-45c1-bcba-d666548a767a None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:58.245749 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048de07f-ecee-45c1-bcba-d666548a767a None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:58.247126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 702/1393] 10.222.0.59 () {54 vars in 967 bytes} [Fri Aug 26 00:46:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:46:58.252308 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e2018-2938-41d3-b506-a227105a24d2 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:58.252630 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e2018-2938-41d3-b506-a227105a24d2 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:58.252909 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e2018-2938-41d3-b506-a227105a24d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:58.329287 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-1e4e2018-2938-41d3-b506-a227105a24d2 None None] MFA Rules not processed for user `abed7ebb0aae4e4693ce08e51a149312`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:46:58.330400 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-1e4e2018-2938-41d3-b506-a227105a24d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:58.365734 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048de07f-ecee-45c1-bcba-d666548a767a None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:46:58.367482 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-048de07f-ecee-45c1-bcba-d666548a767a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:58.379044 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:58.381927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:58.407492 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 692/1394] 10.222.0.59 () {58 vars in 1064 bytes} [Fri Aug 26 00:46:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:46:58.415231 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e2018-2938-41d3-b506-a227105a24d2 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:46:58.423520 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e2018-2938-41d3-b506-a227105a24d2 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:46:58.498941 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016aa940-ab1e-402f-b8c9-96c5dd002591 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abed7ebb0aae4e4693ce08e51a149312', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:46:58.499570 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016aa940-ab1e-402f-b8c9-96c5dd002591 service glance] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:58.499845 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016aa940-ab1e-402f-b8c9-96c5dd002591 service glance] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:58.500111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016aa940-ab1e-402f-b8c9-96c5dd002591 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:58.501730 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-016aa940-ab1e-402f-b8c9-96c5dd002591 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:58.521338 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048de07f-ecee-45c1-bcba-d666548a767a None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:46:58.610350 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:58.611882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016aa940-ab1e-402f-b8c9-96c5dd002591 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:46:58.613374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016aa940-ab1e-402f-b8c9-96c5dd002591 service glance] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:46:58.627941 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:58.632329 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 693/1395] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:46:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:46:58.637613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab616b4-d7eb-40cb-b47b-b5e0367b7a40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:46:58.638222 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab616b4-d7eb-40cb-b47b-b5e0367b7a40 service nova] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:58.638501 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab616b4-d7eb-40cb-b47b-b5e0367b7a40 service nova] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:58.638772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab616b4-d7eb-40cb-b47b-b5e0367b7a40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:58.640382 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-fab616b4-d7eb-40cb-b47b-b5e0367b7a40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:58.730082 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab616b4-d7eb-40cb-b47b-b5e0367b7a40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:46:58.731673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab616b4-d7eb-40cb-b47b-b5e0367b7a40 service nova] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:46:58.749926 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 703/1396] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:46:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:46:59.545195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658ccd7-3fa4-4a5e-9a57-af43c4769734 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:59.545649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658ccd7-3fa4-4a5e-9a57-af43c4769734 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:59.545756 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658ccd7-3fa4-4a5e-9a57-af43c4769734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:59.612449 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-e658ccd7-3fa4-4a5e-9a57-af43c4769734 None None] MFA Rules not processed for user `bbd9bbb083ab4f5e84e135ec5103cdae`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:46:59.613695 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-e658ccd7-3fa4-4a5e-9a57-af43c4769734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:59.699054 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 694/1397] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Aug 26 00:46:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:46:59.744674 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d13ebf-7a8f-495a-a88a-f5ce92febe21 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:59.745130 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d13ebf-7a8f-495a-a88a-f5ce92febe21 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:59.745276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d13ebf-7a8f-495a-a88a-f5ce92febe21 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:59.746622 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 704/1398] 10.222.0.59 () {54 vars in 943 bytes} [Fri Aug 26 00:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:46:59.752912 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb03f8b-ee47-4854-ac09-dc33819a5d89 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:46:59.753200 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb03f8b-ee47-4854-ac09-dc33819a5d89 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:46:59.753504 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb03f8b-ee47-4854-ac09-dc33819a5d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:46:59.830498 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-edb03f8b-ee47-4854-ac09-dc33819a5d89 None None] MFA Rules not processed for user `2f700502671446c797a566f07398a9fc`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:46:59.831698 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-edb03f8b-ee47-4854-ac09-dc33819a5d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:46:59.860821 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:46:59.939631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 695/1399] 10.222.0.59 () {58 vars in 1040 bytes} [Fri Aug 26 00:46:59 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:46:59.949700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d13ebf-7a8f-495a-a88a-f5ce92febe21 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:46:59.951244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-85d13ebf-7a8f-495a-a88a-f5ce92febe21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:00.052275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:00.094522 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d13ebf-7a8f-495a-a88a-f5ce92febe21 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:47:00.216571 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70156b0-6b59-49ad-b133-384ce46033dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f700502671446c797a566f07398a9fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:47:00.217178 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70156b0-6b59-49ad-b133-384ce46033dd service swift] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:00.217454 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70156b0-6b59-49ad-b133-384ce46033dd service swift] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:00.217724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70156b0-6b59-49ad-b133-384ce46033dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:00.219420 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-d70156b0-6b59-49ad-b133-384ce46033dd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:00.301231 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70156b0-6b59-49ad-b133-384ce46033dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:47:00.302812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70156b0-6b59-49ad-b133-384ce46033dd service swift] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:47:00.306239 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:00.311683 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 705/1400] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Aug 26 00:46:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 00:47:00.849455 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac6d45-6743-4edd-a961-75ce063ccdd4 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:00.850049 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac6d45-6743-4edd-a961-75ce063ccdd4 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:00.850049 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac6d45-6743-4edd-a961-75ce063ccdd4 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:00.852510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 696/1401] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:47:00.857701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca5f2c-b6f2-4977-b5de-e9f8e235a456 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:00.858193 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca5f2c-b6f2-4977-b5de-e9f8e235a456 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:00.858280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca5f2c-b6f2-4977-b5de-e9f8e235a456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:00.929818 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-0cca5f2c-b6f2-4977-b5de-e9f8e235a456 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:47:00.931045 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-0cca5f2c-b6f2-4977-b5de-e9f8e235a456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:01.040698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 706/1402] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:47:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:47:01.322050 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac6d45-6743-4edd-a961-75ce063ccdd4 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:47:01.324122 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-28ac6d45-6743-4edd-a961-75ce063ccdd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:01.494913 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac6d45-6743-4edd-a961-75ce063ccdd4 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:47:01.498183 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:01.630052 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499bea53-6073-48d6-a101-e8cda7b31ab4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:47:01.630585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bea53-6073-48d6-a101-e8cda7b31ab4 service nova] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:01.630862 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bea53-6073-48d6-a101-e8cda7b31ab4 service nova] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:01.631131 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bea53-6073-48d6-a101-e8cda7b31ab4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:01.632953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-499bea53-6073-48d6-a101-e8cda7b31ab4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:01.735183 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499bea53-6073-48d6-a101-e8cda7b31ab4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:47:01.736833 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499bea53-6073-48d6-a101-e8cda7b31ab4 service nova] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:47:01.756349 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 697/1403] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:47:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:47:03.938276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be622204-8b14-4220-8f97-cfe4b8deef3b None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:03.938911 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be622204-8b14-4220-8f97-cfe4b8deef3b None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:03.938911 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be622204-8b14-4220-8f97-cfe4b8deef3b None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:03.940392 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 707/1404] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:47:03.945779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767df521-2700-4ab4-96fa-70e6ecb562fb None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:03.946229 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767df521-2700-4ab4-96fa-70e6ecb562fb None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:03.946328 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767df521-2700-4ab4-96fa-70e6ecb562fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:03.957288 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:04.016525 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-767df521-2700-4ab4-96fa-70e6ecb562fb None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:47:04.017816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-767df521-2700-4ab4-96fa-70e6ecb562fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:04.121642 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 698/1405] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:47:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:47:04.418755 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be622204-8b14-4220-8f97-cfe4b8deef3b None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:47:04.420321 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-be622204-8b14-4220-8f97-cfe4b8deef3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:04.517366 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:04.590085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be622204-8b14-4220-8f97-cfe4b8deef3b None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:47:04.719645 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2d8e9-a4de-4d80-adb6-7da3ede32257 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:47:04.720254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2d8e9-a4de-4d80-adb6-7da3ede32257 service nova] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:04.720563 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2d8e9-a4de-4d80-adb6-7da3ede32257 service nova] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:04.720835 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2d8e9-a4de-4d80-adb6-7da3ede32257 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:04.722452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-55e2d8e9-a4de-4d80-adb6-7da3ede32257 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:04.767629 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:04.825716 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2d8e9-a4de-4d80-adb6-7da3ede32257 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:47:04.827386 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2d8e9-a4de-4d80-adb6-7da3ede32257 service nova] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:47:04.851218 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 708/1406] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:47:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:47:06.925918 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bda07-cfdf-450e-826e-00fdd62a5b8c None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:06.926501 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bda07-cfdf-450e-826e-00fdd62a5b8c None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:06.926501 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bda07-cfdf-450e-826e-00fdd62a5b8c None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:06.927956 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 699/1407] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:47:06.934530 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d79d1-6a6a-470a-99a4-4b54cd18a855 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:06.935458 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d79d1-6a6a-470a-99a4-4b54cd18a855 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:06.935739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d79d1-6a6a-470a-99a4-4b54cd18a855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:07.014574 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-b74d79d1-6a6a-470a-99a4-4b54cd18a855 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:47:07.015823 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-b74d79d1-6a6a-470a-99a4-4b54cd18a855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:07.128628 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 709/1408] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:47:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:47:07.422887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73bda07-cfdf-450e-826e-00fdd62a5b8c None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:47:07.424457 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-f73bda07-cfdf-450e-826e-00fdd62a5b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:07.543430 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:07.631460 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73bda07-cfdf-450e-826e-00fdd62a5b8c None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:47:07.782669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6fe2e-ce92-48c0-abb2-faa38de7ffb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:47:07.783218 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6fe2e-ce92-48c0-abb2-faa38de7ffb8 service nova] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:07.783499 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6fe2e-ce92-48c0-abb2-faa38de7ffb8 service nova] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:07.783840 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6fe2e-ce92-48c0-abb2-faa38de7ffb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:07.785517 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-2df6fe2e-ce92-48c0-abb2-faa38de7ffb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:07.845810 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:07.905123 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6fe2e-ce92-48c0-abb2-faa38de7ffb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:47:07.905123 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6fe2e-ce92-48c0-abb2-faa38de7ffb8 service nova] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:47:07.917790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 700/1409] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:47:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:47:17.122717 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=93 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:18.556001 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=24 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:23.760659 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471ea94-ca28-4621-92cd-38785826735c None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:23.761223 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471ea94-ca28-4621-92cd-38785826735c None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:23.761223 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471ea94-ca28-4621-92cd-38785826735c None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:23.762470 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 710/1410] 10.222.0.59 () {54 vars in 918 bytes} [Fri Aug 26 00:47:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:47:23.767450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd577a52-f9a6-4289-92af-7b79f74a658e None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:23.767885 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd577a52-f9a6-4289-92af-7b79f74a658e None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:23.768002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd577a52-f9a6-4289-92af-7b79f74a658e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:23.843954 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-dd577a52-f9a6-4289-92af-7b79f74a658e None None] MFA Rules not processed for user `2876f4182f0949039b862f053533fe85`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:47:23.845196 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-dd577a52-f9a6-4289-92af-7b79f74a658e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:23.965947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 701/1411] 10.222.0.59 () {58 vars in 1015 bytes} [Fri Aug 26 00:47:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:47:23.979852 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9fea4a-20f5-48ad-85eb-27d8daa5c4b2 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:23.980135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9fea4a-20f5-48ad-85eb-27d8daa5c4b2 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:23.980408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9fea4a-20f5-48ad-85eb-27d8daa5c4b2 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:23.981778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 711/1412] 10.222.0.59 () {54 vars in 918 bytes} [Fri Aug 26 00:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:47:23.986518 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41197ae-ca64-4bb7-84ee-2b8ab3191096 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:23.986807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41197ae-ca64-4bb7-84ee-2b8ab3191096 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:23.987080 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41197ae-ca64-4bb7-84ee-2b8ab3191096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:24.061050 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:24.063591 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-f41197ae-ca64-4bb7-84ee-2b8ab3191096 None None] MFA Rules not processed for user `0602593263ca4b05a657d5b7697f932f`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:47:24.064816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-f41197ae-ca64-4bb7-84ee-2b8ab3191096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:24.176432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 702/1413] 10.222.0.59 () {58 vars in 1015 bytes} [Fri Aug 26 00:47:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:47:26.565219 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ccd26-5fa4-421d-8519-ca54134d4caf None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:26.565767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ccd26-5fa4-421d-8519-ca54134d4caf None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:26.565878 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ccd26-5fa4-421d-8519-ca54134d4caf None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:26.567125 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 00:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:47:26.573237 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731c091-a310-4039-8a28-b1f95835cf71 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:26.573653 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731c091-a310-4039-8a28-b1f95835cf71 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:26.573790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731c091-a310-4039-8a28-b1f95835cf71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:26.638519 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-6731c091-a310-4039-8a28-b1f95835cf71 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:47:26.639719 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-6731c091-a310-4039-8a28-b1f95835cf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:26.693563 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:26.733802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 00:47:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:47:27.022914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ccd26-5fa4-421d-8519-ca54134d4caf None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:47:27.024497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-653ccd26-5fa4-421d-8519-ca54134d4caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:27.171349 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ccd26-5fa4-421d-8519-ca54134d4caf None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:47:27.243423 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:27.282980 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a21aa-d364-4296-8138-2111192f5007 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:47:27.283522 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a21aa-d364-4296-8138-2111192f5007 service nova] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:27.283802 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a21aa-d364-4296-8138-2111192f5007 service nova] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:27.284073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a21aa-d364-4296-8138-2111192f5007 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:27.285740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-273a21aa-d364-4296-8138-2111192f5007 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:27.384892 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a21aa-d364-4296-8138-2111192f5007 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:47:27.386484 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a21aa-d364-4296-8138-2111192f5007 service nova] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:47:27.404494 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 713/1416] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:47:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:47:28.301440 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83c305-896e-4a6f-99e2-9a5c7915b1c5 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:28.301967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83c305-896e-4a6f-99e2-9a5c7915b1c5 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:28.301967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83c305-896e-4a6f-99e2-9a5c7915b1c5 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:28.303335 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 00:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:47:28.309122 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf6c38-e8a6-4e74-848c-76bbcaa421e7 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:28.309550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf6c38-e8a6-4e74-848c-76bbcaa421e7 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:28.309656 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf6c38-e8a6-4e74-848c-76bbcaa421e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:28.385491 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-d3cf6c38-e8a6-4e74-848c-76bbcaa421e7 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:47:28.386697 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-d3cf6c38-e8a6-4e74-848c-76bbcaa421e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:28.413581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:28.483720 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 00:47:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:47:28.637555 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac83c305-896e-4a6f-99e2-9a5c7915b1c5 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:47:28.639110 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-ac83c305-896e-4a6f-99e2-9a5c7915b1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:28.787264 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac83c305-896e-4a6f-99e2-9a5c7915b1c5 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:47:28.827497 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:28.900754 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b4212e-779f-48f4-8816-b54df4eb1d6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:47:28.901303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4212e-779f-48f4-8816-b54df4eb1d6e service nova] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:28.901574 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4212e-779f-48f4-8816-b54df4eb1d6e service nova] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:28.901843 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4212e-779f-48f4-8816-b54df4eb1d6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:28.903442 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-13b4212e-779f-48f4-8816-b54df4eb1d6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:29.000447 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b4212e-779f-48f4-8816-b54df4eb1d6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:47:29.002117 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b4212e-779f-48f4-8816-b54df4eb1d6e service nova] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:47:29.021150 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 705/1419] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:47:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:47:29.901171 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac5b27-1952-4106-a4c0-4f7be1ad814b None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:29.901745 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac5b27-1952-4106-a4c0-4f7be1ad814b None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:29.901745 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac5b27-1952-4106-a4c0-4f7be1ad814b None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:29.903131 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 00:47:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:47:29.908164 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab7950-e4f6-48c2-b1b4-4e66eec8fdd3 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:29.908599 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab7950-e4f6-48c2-b1b4-4e66eec8fdd3 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:29.908737 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab7950-e4f6-48c2-b1b4-4e66eec8fdd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:29.960770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:29.982777 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-40ab7950-e4f6-48c2-b1b4-4e66eec8fdd3 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:47:29.983978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-40ab7950-e4f6-48c2-b1b4-4e66eec8fdd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:30.079561 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 00:47:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:47:30.233552 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac5b27-1952-4106-a4c0-4f7be1ad814b None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:47:30.235076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-60ac5b27-1952-4106-a4c0-4f7be1ad814b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:30.379168 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac5b27-1952-4106-a4c0-4f7be1ad814b None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:47:30.391377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:30.491727 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f82eb75-1f63-4b2b-be08-58241796e53a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:47:30.492274 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82eb75-1f63-4b2b-be08-58241796e53a service nova] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:30.492584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82eb75-1f63-4b2b-be08-58241796e53a service nova] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:30.492860 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82eb75-1f63-4b2b-be08-58241796e53a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:30.494493 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-0f82eb75-1f63-4b2b-be08-58241796e53a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:30.583706 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f82eb75-1f63-4b2b-be08-58241796e53a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:47:30.585336 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f82eb75-1f63-4b2b-be08-58241796e53a service nova] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:47:30.603513 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 716/1422] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:47:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:47:31.458312 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b28dc-3a15-4066-8322-f55760de3830 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:31.458755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b28dc-3a15-4066-8322-f55760de3830 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:31.458868 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b28dc-3a15-4066-8322-f55760de3830 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:31.460200 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 00:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:47:31.465204 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3e5d1-2858-4b89-985d-86e69ec9707d None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:31.465610 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3e5d1-2858-4b89-985d-86e69ec9707d None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:31.465759 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3e5d1-2858-4b89-985d-86e69ec9707d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:31.485668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:31.548306 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-2fa3e5d1-2858-4b89-985d-86e69ec9707d None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:47:31.549570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-2fa3e5d1-2858-4b89-985d-86e69ec9707d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:31.644333 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 00:47:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:47:31.793673 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b28dc-3a15-4066-8322-f55760de3830 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:47:31.795158 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-ac7b28dc-3a15-4066-8322-f55760de3830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:31.902656 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:31.939098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b28dc-3a15-4066-8322-f55760de3830 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:47:32.049439 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39af68-ba58-4c1d-987d-30d6d0a13769 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:47:32.049957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39af68-ba58-4c1d-987d-30d6d0a13769 service nova] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:32.050237 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39af68-ba58-4c1d-987d-30d6d0a13769 service nova] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:32.050506 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39af68-ba58-4c1d-987d-30d6d0a13769 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:32.052105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-fe39af68-ba58-4c1d-987d-30d6d0a13769 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:32.125988 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:32.139471 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe39af68-ba58-4c1d-987d-30d6d0a13769 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:47:32.141102 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe39af68-ba58-4c1d-987d-30d6d0a13769 service nova] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:47:32.159335 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 708/1425] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:47:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:47:33.024038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec06b3-bb90-4e4e-a2b4-04e6d591a0b6 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:33.024606 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec06b3-bb90-4e4e-a2b4-04e6d591a0b6 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:33.024707 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec06b3-bb90-4e4e-a2b4-04e6d591a0b6 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:33.026114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 00:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:47:33.031170 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d5c5d-5fa9-47a3-b980-5bf79ef2b75e None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:33.031567 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d5c5d-5fa9-47a3-b980-5bf79ef2b75e None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:33.031743 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d5c5d-5fa9-47a3-b980-5bf79ef2b75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:33.109666 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-189d5c5d-5fa9-47a3-b980-5bf79ef2b75e None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:47:33.110878 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-189d5c5d-5fa9-47a3-b980-5bf79ef2b75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:33.207080 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 00:47:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:47:33.360123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec06b3-bb90-4e4e-a2b4-04e6d591a0b6 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:47:33.361710 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-75ec06b3-bb90-4e4e-a2b4-04e6d591a0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:33.444360 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:33.505330 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec06b3-bb90-4e4e-a2b4-04e6d591a0b6 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:47:33.616899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c57fae-1b70-4f19-88f7-9e6c48d65aff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:47:33.617439 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c57fae-1b70-4f19-88f7-9e6c48d65aff service nova] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:33.617715 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c57fae-1b70-4f19-88f7-9e6c48d65aff service nova] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:33.617986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c57fae-1b70-4f19-88f7-9e6c48d65aff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:33.619589 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-c8c57fae-1b70-4f19-88f7-9e6c48d65aff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:33.667195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:33.707224 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c57fae-1b70-4f19-88f7-9e6c48d65aff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:47:33.708911 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c57fae-1b70-4f19-88f7-9e6c48d65aff service nova] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:47:33.727158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 719/1428] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:47:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:47:34.644685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0d707-e0ed-469d-a177-979ffb709657 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:34.645217 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0d707-e0ed-469d-a177-979ffb709657 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:34.645217 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0d707-e0ed-469d-a177-979ffb709657 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:34.646474 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 00:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:47:34.651408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ba362-a08d-465d-beac-2f6ac47c36c3 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:34.651858 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ba362-a08d-465d-beac-2f6ac47c36c3 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:34.651974 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ba362-a08d-465d-beac-2f6ac47c36c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:34.728583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-0c3ba362-a08d-465d-beac-2f6ac47c36c3 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:47:34.729843 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-0c3ba362-a08d-465d-beac-2f6ac47c36c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:34.825154 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 00:47:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:47:34.974307 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0d707-e0ed-469d-a177-979ffb709657 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:47:34.975747 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-47c0d707-e0ed-469d-a177-979ffb709657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:35.024199 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:35.121707 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0d707-e0ed-469d-a177-979ffb709657 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:47:35.231679 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06044f-9769-46fa-9ad8-e78b74a80a9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:47:35.232228 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06044f-9769-46fa-9ad8-e78b74a80a9e service nova] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:35.232525 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06044f-9769-46fa-9ad8-e78b74a80a9e service nova] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:35.232792 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06044f-9769-46fa-9ad8-e78b74a80a9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:35.234382 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-dd06044f-9769-46fa-9ad8-e78b74a80a9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:35.241084 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:35.321699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06044f-9769-46fa-9ad8-e78b74a80a9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:47:35.323294 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06044f-9769-46fa-9ad8-e78b74a80a9e service nova] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:47:35.341751 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 711/1431] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:47:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:47:36.275994 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd536523-2e39-477a-a0c6-c6dec73b5c42 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:36.276556 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd536523-2e39-477a-a0c6-c6dec73b5c42 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:36.276556 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd536523-2e39-477a-a0c6-c6dec73b5c42 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:36.277836 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 00:47:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:47:36.282865 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c3843-4dd7-497f-a614-2852d5ff20c1 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:36.283296 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c3843-4dd7-497f-a614-2852d5ff20c1 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:36.283408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c3843-4dd7-497f-a614-2852d5ff20c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:36.356675 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-572c3843-4dd7-497f-a614-2852d5ff20c1 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:47:36.357867 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-572c3843-4dd7-497f-a614-2852d5ff20c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:36.426096 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:36.455212 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 00:47:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:47:36.606625 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd536523-2e39-477a-a0c6-c6dec73b5c42 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:47:36.608193 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-fd536523-2e39-477a-a0c6-c6dec73b5c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:36.625768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:36.751904 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd536523-2e39-477a-a0c6-c6dec73b5c42 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:47:36.838886 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:36.864365 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac38e4-d389-434c-929d-8e33b910c663 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:47:36.864943 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac38e4-d389-434c-929d-8e33b910c663 service nova] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:36.865225 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac38e4-d389-434c-929d-8e33b910c663 service nova] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:36.865499 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac38e4-d389-434c-929d-8e33b910c663 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:36.867114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-e0ac38e4-d389-434c-929d-8e33b910c663 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:36.956704 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac38e4-d389-434c-929d-8e33b910c663 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:47:36.958294 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac38e4-d389-434c-929d-8e33b910c663 service nova] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:47:36.975779 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 722/1434] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:47:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:47:37.828749 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce474863-ed39-470d-9526-0a7d5991d107 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:37.829319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce474863-ed39-470d-9526-0a7d5991d107 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:37.829319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce474863-ed39-470d-9526-0a7d5991d107 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:37.830710 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 00:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:47:37.835775 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50479841-a531-471d-ba61-d23dc70ec53d None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:37.836639 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50479841-a531-471d-ba61-d23dc70ec53d None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:37.837136 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50479841-a531-471d-ba61-d23dc70ec53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:37.913677 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-50479841-a531-471d-ba61-d23dc70ec53d None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:47:37.914873 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-50479841-a531-471d-ba61-d23dc70ec53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:37.956349 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:38.009644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 00:47:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:47:38.163585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce474863-ed39-470d-9526-0a7d5991d107 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:47:38.165186 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-ce474863-ed39-470d-9526-0a7d5991d107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:38.308927 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce474863-ed39-470d-9526-0a7d5991d107 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:47:38.360985 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:38.421708 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52f4da-3070-469c-8dd9-041019bd779d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:47:38.422417 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52f4da-3070-469c-8dd9-041019bd779d service nova] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:38.422698 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52f4da-3070-469c-8dd9-041019bd779d service nova] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:38.422972 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52f4da-3070-469c-8dd9-041019bd779d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:38.424605 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-8d52f4da-3070-469c-8dd9-041019bd779d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:38.514077 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d52f4da-3070-469c-8dd9-041019bd779d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:47:38.515674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d52f4da-3070-469c-8dd9-041019bd779d service nova] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:47:38.533681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 714/1437] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:47:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:47:39.408021 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b189c60-d1b1-4c26-847c-6aa3d1e09781 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:39.408688 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b189c60-d1b1-4c26-847c-6aa3d1e09781 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:39.408688 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b189c60-d1b1-4c26-847c-6aa3d1e09781 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:39.409939 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 00:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:47:39.414880 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626cb8d-acad-4da6-9d3c-c5cefb12418d None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:39.415290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626cb8d-acad-4da6-9d3c-c5cefb12418d None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:39.415435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626cb8d-acad-4da6-9d3c-c5cefb12418d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:39.490434 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-5626cb8d-acad-4da6-9d3c-c5cefb12418d None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:47:39.491647 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-5626cb8d-acad-4da6-9d3c-c5cefb12418d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:39.493658 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:39.587587 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 00:47:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:47:39.737702 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b189c60-d1b1-4c26-847c-6aa3d1e09781 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:47:39.739122 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-1b189c60-d1b1-4c26-847c-6aa3d1e09781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:39.883285 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b189c60-d1b1-4c26-847c-6aa3d1e09781 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:47:39.910247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:39.995671 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4351c9-4f57-460d-8a11-7c2525aaf88e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:47:39.996213 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4351c9-4f57-460d-8a11-7c2525aaf88e service nova] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:39.996515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4351c9-4f57-460d-8a11-7c2525aaf88e service nova] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:39.996794 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4351c9-4f57-460d-8a11-7c2525aaf88e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:39.998483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-1c4351c9-4f57-460d-8a11-7c2525aaf88e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:40.090269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4351c9-4f57-460d-8a11-7c2525aaf88e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:47:40.091856 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4351c9-4f57-460d-8a11-7c2525aaf88e service nova] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:47:40.109125 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 725/1440] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:47:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:47:41.960067 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd368b06-27fe-4043-bc8c-473fbc4efda8 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:41.960662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd368b06-27fe-4043-bc8c-473fbc4efda8 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:41.960662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd368b06-27fe-4043-bc8c-473fbc4efda8 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:41.961989 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 00:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:47:41.968072 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191d0ac-0e08-413d-99c7-7cebc11f5fcc None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:41.968571 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191d0ac-0e08-413d-99c7-7cebc11f5fcc None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:41.968684 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191d0ac-0e08-413d-99c7-7cebc11f5fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:42.009161 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:42.034465 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-1191d0ac-0e08-413d-99c7-7cebc11f5fcc None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:47:42.035660 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-1191d0ac-0e08-413d-99c7-7cebc11f5fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:42.130312 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 00:47:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:47:42.413204 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd368b06-27fe-4043-bc8c-473fbc4efda8 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:47:42.414765 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-cd368b06-27fe-4043-bc8c-473fbc4efda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:42.517636 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd368b06-27fe-4043-bc8c-473fbc4efda8 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:47:42.549421 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:42.590055 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84fae61-c0a0-4268-b3d1-b01c346c0c81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:47:42.590661 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fae61-c0a0-4268-b3d1-b01c346c0c81 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:42.590939 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fae61-c0a0-4268-b3d1-b01c346c0c81 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:42.591211 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fae61-c0a0-4268-b3d1-b01c346c0c81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:42.592835 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-b84fae61-c0a0-4268-b3d1-b01c346c0c81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:42.698163 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84fae61-c0a0-4268-b3d1-b01c346c0c81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:47:42.699529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84fae61-c0a0-4268-b3d1-b01c346c0c81 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:47:42.717610 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 717/1443] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:47:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:47:44.778291 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6129a8-6973-4892-a98f-eb908e91e232 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:44.778851 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6129a8-6973-4892-a98f-eb908e91e232 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:44.778851 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6129a8-6973-4892-a98f-eb908e91e232 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:44.780184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 00:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:47:44.785900 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826388e-91e1-41cf-b7ff-c11d358bc88a None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:44.786332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826388e-91e1-41cf-b7ff-c11d358bc88a None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:44.786446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826388e-91e1-41cf-b7ff-c11d358bc88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:44.856904 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-c826388e-91e1-41cf-b7ff-c11d358bc88a None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:47:44.858194 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-c826388e-91e1-41cf-b7ff-c11d358bc88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:44.871199 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:44.953750 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 00:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:47:45.232285 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6129a8-6973-4892-a98f-eb908e91e232 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:47:45.233888 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-cf6129a8-6973-4892-a98f-eb908e91e232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:45.338930 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6129a8-6973-4892-a98f-eb908e91e232 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:47:45.350961 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:45.411799 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f46c20-df6a-41ea-b931-194887d37b85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:47:45.412415 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f46c20-df6a-41ea-b931-194887d37b85 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:45.412727 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f46c20-df6a-41ea-b931-194887d37b85 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:45.413003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f46c20-df6a-41ea-b931-194887d37b85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:45.414635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a1f46c20-df6a-41ea-b931-194887d37b85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:45.521936 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f46c20-df6a-41ea-b931-194887d37b85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:47:45.523294 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f46c20-df6a-41ea-b931-194887d37b85 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:47:45.542308 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 728/1446] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:47:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:47:47.879927 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b3915-6c0a-4a64-affb-6a54a7473549 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:47.880556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b3915-6c0a-4a64-affb-6a54a7473549 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:47.880556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b3915-6c0a-4a64-affb-6a54a7473549 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:47.881862 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 00:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:47:47.887646 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020a45a-9478-4ec7-aa22-d7eaaa3e370d None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:47.888058 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020a45a-9478-4ec7-aa22-d7eaaa3e370d None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:47.888208 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020a45a-9478-4ec7-aa22-d7eaaa3e370d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:47.937026 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:47.955342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-0020a45a-9478-4ec7-aa22-d7eaaa3e370d None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:47:47.956593 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-0020a45a-9478-4ec7-aa22-d7eaaa3e370d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:48.052109 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 00:47:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:47:48.336776 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b3915-6c0a-4a64-affb-6a54a7473549 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:47:48.338408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-996b3915-6c0a-4a64-affb-6a54a7473549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:48.440632 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b3915-6c0a-4a64-affb-6a54a7473549 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:47:48.482857 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:48.516189 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2bf0a-a9ed-49f4-b361-14827c693e42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:47:48.516823 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2bf0a-a9ed-49f4-b361-14827c693e42 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:48.517101 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2bf0a-a9ed-49f4-b361-14827c693e42 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:48.517410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2bf0a-a9ed-49f4-b361-14827c693e42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:48.519717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-82f2bf0a-a9ed-49f4-b361-14827c693e42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:48.625889 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2bf0a-a9ed-49f4-b361-14827c693e42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:47:48.627242 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2bf0a-a9ed-49f4-b361-14827c693e42 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:47:48.645626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 720/1449] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:47:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:47:50.694341 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8819c-ef1d-4baf-a711-b538a27e4ba5 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:50.694884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8819c-ef1d-4baf-a711-b538a27e4ba5 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:50.694884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8819c-ef1d-4baf-a711-b538a27e4ba5 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:50.696188 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 00:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:47:50.702041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62cfe68-2c27-4bfe-9583-57c3bc30abc5 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:50.702454 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62cfe68-2c27-4bfe-9583-57c3bc30abc5 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:50.702593 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62cfe68-2c27-4bfe-9583-57c3bc30abc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:50.766863 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-c62cfe68-2c27-4bfe-9583-57c3bc30abc5 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:47:50.768065 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-c62cfe68-2c27-4bfe-9583-57c3bc30abc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:50.788427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:50.862200 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 00:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:47:51.147956 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8819c-ef1d-4baf-a711-b538a27e4ba5 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:47:51.149554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-c8f8819c-ef1d-4baf-a711-b538a27e4ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:51.252079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8819c-ef1d-4baf-a711-b538a27e4ba5 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:47:51.271310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:51.326753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658bb66f-8b8a-4a44-a705-524e0c325f80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:47:51.327391 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bb66f-8b8a-4a44-a705-524e0c325f80 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:51.327672 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bb66f-8b8a-4a44-a705-524e0c325f80 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:51.327952 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bb66f-8b8a-4a44-a705-524e0c325f80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:51.329786 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-658bb66f-8b8a-4a44-a705-524e0c325f80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:51.443614 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658bb66f-8b8a-4a44-a705-524e0c325f80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:47:51.445123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658bb66f-8b8a-4a44-a705-524e0c325f80 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:47:51.464359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 731/1452] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:47:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:47:53.423042 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07db3f-efb9-44fd-8f04-917ebc830dd5 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:53.423576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07db3f-efb9-44fd-8f04-917ebc830dd5 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:53.423576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07db3f-efb9-44fd-8f04-917ebc830dd5 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:53.424981 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 00:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:47:53.430333 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c85f0c-23bc-4c05-96fc-68c8846bd9b7 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:53.430735 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c85f0c-23bc-4c05-96fc-68c8846bd9b7 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:53.430907 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c85f0c-23bc-4c05-96fc-68c8846bd9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:53.495764 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:53.498737 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-70c85f0c-23bc-4c05-96fc-68c8846bd9b7 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:47:53.499946 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-70c85f0c-23bc-4c05-96fc-68c8846bd9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:53.595829 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 00:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:47:53.878549 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07db3f-efb9-44fd-8f04-917ebc830dd5 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:47:53.880084 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-ea07db3f-efb9-44fd-8f04-917ebc830dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:53.980464 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07db3f-efb9-44fd-8f04-917ebc830dd5 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:47:54.029501 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:54.053659 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9b54d-61ca-4d89-9004-42140cd2e1a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:47:54.055875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9b54d-61ca-4d89-9004-42140cd2e1a5 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:54.055875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9b54d-61ca-4d89-9004-42140cd2e1a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:54.055875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9b54d-61ca-4d89-9004-42140cd2e1a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:54.056387 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-2ec9b54d-61ca-4d89-9004-42140cd2e1a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:54.162000 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9b54d-61ca-4d89-9004-42140cd2e1a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:47:54.163340 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9b54d-61ca-4d89-9004-42140cd2e1a5 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:47:54.181823 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 723/1455] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:47:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:47:56.190160 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307fd4c-cd28-4b30-b600-ddb7198f8f56 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:56.190734 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307fd4c-cd28-4b30-b600-ddb7198f8f56 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:56.190734 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307fd4c-cd28-4b30-b600-ddb7198f8f56 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:56.192038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 00:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:47:56.197268 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cb92c-3190-4df7-b422-2c511cb47a92 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:56.197697 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cb92c-3190-4df7-b422-2c511cb47a92 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:56.197812 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cb92c-3190-4df7-b422-2c511cb47a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:56.263215 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-273cb92c-3190-4df7-b422-2c511cb47a92 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:47:56.264421 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-273cb92c-3190-4df7-b422-2c511cb47a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:56.293781 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:56.360598 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 00:47:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:47:56.641337 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307fd4c-cd28-4b30-b600-ddb7198f8f56 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:47:56.642882 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-1307fd4c-cd28-4b30-b600-ddb7198f8f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:56.744860 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307fd4c-cd28-4b30-b600-ddb7198f8f56 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:47:56.773502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:56.819111 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452416af-79de-4bc2-b35a-b488b398ce9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:47:56.819713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452416af-79de-4bc2-b35a-b488b398ce9c service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:56.819991 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452416af-79de-4bc2-b35a-b488b398ce9c service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:56.820252 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452416af-79de-4bc2-b35a-b488b398ce9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:56.821871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-452416af-79de-4bc2-b35a-b488b398ce9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:56.929961 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452416af-79de-4bc2-b35a-b488b398ce9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:47:56.931327 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452416af-79de-4bc2-b35a-b488b398ce9c service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:47:56.950037 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 734/1458] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:47:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:47:58.924275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4380c7-a61b-4e35-9a88-26efba17aecf None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:58.924898 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4380c7-a61b-4e35-9a88-26efba17aecf None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:58.924898 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4380c7-a61b-4e35-9a88-26efba17aecf None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:58.926214 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 00:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:47:58.932566 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbac2e-9172-4bd3-82d5-004782808502 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:58.933019 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbac2e-9172-4bd3-82d5-004782808502 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:58.933148 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbac2e-9172-4bd3-82d5-004782808502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:59.000801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-f6bbac2e-9172-4bd3-82d5-004782808502 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:47:59.002015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f6bbac2e-9172-4bd3-82d5-004782808502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:59.010267 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:59.099242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 00:47:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:47:59.380806 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4380c7-a61b-4e35-9a88-26efba17aecf None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:47:59.382409 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-0e4380c7-a61b-4e35-9a88-26efba17aecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:59.485132 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4380c7-a61b-4e35-9a88-26efba17aecf None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:47:59.539300 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:47:59.557748 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518283c-d4da-4382-bcc3-c702035c17b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:47:59.558352 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518283c-d4da-4382-bcc3-c702035c17b4 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:47:59.558628 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518283c-d4da-4382-bcc3-c702035c17b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:47:59.558952 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518283c-d4da-4382-bcc3-c702035c17b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:47:59.560572 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-9518283c-d4da-4382-bcc3-c702035c17b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:47:59.668194 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9518283c-d4da-4382-bcc3-c702035c17b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:47:59.669573 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9518283c-d4da-4382-bcc3-c702035c17b4 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:47:59.687581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 726/1461] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:47:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:48:01.627400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ca5a9-213e-4edf-ac81-9de3f2289e31 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:01.627912 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ca5a9-213e-4edf-ac81-9de3f2289e31 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:01.628008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ca5a9-213e-4edf-ac81-9de3f2289e31 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:01.629322 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 00:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:48:01.634726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022f30c-d4ee-4fed-a223-5b57043e47c9 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:01.635142 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022f30c-d4ee-4fed-a223-5b57043e47c9 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:01.635288 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022f30c-d4ee-4fed-a223-5b57043e47c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:01.700458 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-8022f30c-d4ee-4fed-a223-5b57043e47c9 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:48:01.701623 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-8022f30c-d4ee-4fed-a223-5b57043e47c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:01.739188 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:48:01.796320 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 00:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:48:02.082678 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ca5a9-213e-4edf-ac81-9de3f2289e31 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:48:02.084223 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-cc1ca5a9-213e-4edf-ac81-9de3f2289e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:02.186617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ca5a9-213e-4edf-ac81-9de3f2289e31 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:48:02.224122 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:48:02.258530 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1c517-efd2-4ccc-973b-0fed4f43a98c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:48:02.259144 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1c517-efd2-4ccc-973b-0fed4f43a98c service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:02.259424 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1c517-efd2-4ccc-973b-0fed4f43a98c service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:02.259699 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1c517-efd2-4ccc-973b-0fed4f43a98c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:02.261874 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-c5d1c517-efd2-4ccc-973b-0fed4f43a98c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:02.369089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1c517-efd2-4ccc-973b-0fed4f43a98c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:48:02.371025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1c517-efd2-4ccc-973b-0fed4f43a98c service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:48:02.389506 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 737/1464] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:48:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:48:04.320328 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a4d00-af09-42f4-94f0-0142ff56a0e6 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:04.320815 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a4d00-af09-42f4-94f0-0142ff56a0e6 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:04.320911 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a4d00-af09-42f4-94f0-0142ff56a0e6 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:04.322268 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 00:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:48:04.328165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb082a-eefc-4252-a5a0-33cbf889d46c None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:04.328612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb082a-eefc-4252-a5a0-33cbf889d46c None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:04.328790 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb082a-eefc-4252-a5a0-33cbf889d46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:04.395830 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-aabb082a-eefc-4252-a5a0-33cbf889d46c None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:48:04.397058 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-aabb082a-eefc-4252-a5a0-33cbf889d46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:04.415859 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:48:04.495970 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 00:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:48:04.777788 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a4d00-af09-42f4-94f0-0142ff56a0e6 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:48:04.779326 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-3f5a4d00-af09-42f4-94f0-0142ff56a0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:04.884435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a4d00-af09-42f4-94f0-0142ff56a0e6 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:48:04.950571 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:48:04.957085 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95164be3-8f56-43f8-a8d8-f2dff36eb98a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:48:04.957689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95164be3-8f56-43f8-a8d8-f2dff36eb98a service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:04.957969 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95164be3-8f56-43f8-a8d8-f2dff36eb98a service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:04.958232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95164be3-8f56-43f8-a8d8-f2dff36eb98a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:04.959844 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-95164be3-8f56-43f8-a8d8-f2dff36eb98a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:05.069975 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95164be3-8f56-43f8-a8d8-f2dff36eb98a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:48:05.071409 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95164be3-8f56-43f8-a8d8-f2dff36eb98a service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:48:05.089089 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 729/1467] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:48:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:48:07.012457 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78831a5-1b06-4a7c-801c-943fe30ac137 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:07.013035 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78831a5-1b06-4a7c-801c-943fe30ac137 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:07.013035 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78831a5-1b06-4a7c-801c-943fe30ac137 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:07.014393 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 00:48:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:48:07.019722 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cae85-5d1b-49f4-bca2-9ea68f0945f9 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:07.020140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cae85-5d1b-49f4-bca2-9ea68f0945f9 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:07.020274 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cae85-5d1b-49f4-bca2-9ea68f0945f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:07.085950 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-978cae85-5d1b-49f4-bca2-9ea68f0945f9 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:48:07.087159 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-978cae85-5d1b-49f4-bca2-9ea68f0945f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:07.134362 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:48:07.181938 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 00:48:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:48:07.473089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78831a5-1b06-4a7c-801c-943fe30ac137 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:48:07.474676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f78831a5-1b06-4a7c-801c-943fe30ac137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:07.579734 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78831a5-1b06-4a7c-801c-943fe30ac137 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:48:07.626940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:48:07.652399 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df6484-ae0c-419b-a383-a0ad15c89230 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:48:07.653029 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df6484-ae0c-419b-a383-a0ad15c89230 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:07.653302 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df6484-ae0c-419b-a383-a0ad15c89230 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:07.653578 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df6484-ae0c-419b-a383-a0ad15c89230 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:07.655151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-25df6484-ae0c-419b-a383-a0ad15c89230 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:07.760785 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df6484-ae0c-419b-a383-a0ad15c89230 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:48:07.762178 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df6484-ae0c-419b-a383-a0ad15c89230 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:48:07.780630 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 740/1470] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:48:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:48:17.183387 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=22 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:48:17.778384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=66 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:48:27.315763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252834c-66ec-48d0-a3f9-932a4f6d3028 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:27.316355 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252834c-66ec-48d0-a3f9-932a4f6d3028 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:27.316355 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252834c-66ec-48d0-a3f9-932a4f6d3028 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:27.317655 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 731/1471] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:48:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:48:27.322848 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7e91a-f62a-4378-8649-454ba824fd60 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:27.323246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7e91a-f62a-4378-8649-454ba824fd60 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:27.323407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7e91a-f62a-4378-8649-454ba824fd60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:27.390088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-c1b7e91a-f62a-4378-8649-454ba824fd60 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:48:27.391301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-c1b7e91a-f62a-4378-8649-454ba824fd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:27.496399 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 741/1472] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:48:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:48:27.792192 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20551513-cdb4-42b3-8964-fadc8fba9f5d None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:27.792506 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20551513-cdb4-42b3-8964-fadc8fba9f5d None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:27.792771 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20551513-cdb4-42b3-8964-fadc8fba9f5d None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:27.794120 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 732/1473] 10.222.0.59 () {54 vars in 967 bytes} [Fri Aug 26 00:48:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:48:27.798823 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c819ee5-08da-494e-ad8c-76d197cedb23 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:27.799105 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c819ee5-08da-494e-ad8c-76d197cedb23 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:27.799378 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c819ee5-08da-494e-ad8c-76d197cedb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:27.874316 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-4c819ee5-08da-494e-ad8c-76d197cedb23 None None] MFA Rules not processed for user `abed7ebb0aae4e4693ce08e51a149312`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:48:27.875456 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-4c819ee5-08da-494e-ad8c-76d197cedb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:27.895812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:48:27.951120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 742/1474] 10.222.0.59 () {58 vars in 1064 bytes} [Fri Aug 26 00:48:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:48:27.958629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20551513-cdb4-42b3-8964-fadc8fba9f5d None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:48:27.960133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-20551513-cdb4-42b3-8964-fadc8fba9f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:28.061682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20551513-cdb4-42b3-8964-fadc8fba9f5d None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:48:28.132310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8779ef02-2f1f-4edf-830d-146ceb00b9de service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abed7ebb0aae4e4693ce08e51a149312', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:48:28.132872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779ef02-2f1f-4edf-830d-146ceb00b9de service glance] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:28.133135 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779ef02-2f1f-4edf-830d-146ceb00b9de service glance] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:28.133404 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779ef02-2f1f-4edf-830d-146ceb00b9de service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:28.134983 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-8779ef02-2f1f-4edf-830d-146ceb00b9de service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:28.187267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:48:28.241200 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8779ef02-2f1f-4edf-830d-146ceb00b9de service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:48:28.242634 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8779ef02-2f1f-4edf-830d-146ceb00b9de service glance] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:48:28.261011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 733/1475] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:48:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:48:30.315362 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162da15-fae7-4c62-a2ae-c2ccbe974cda None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:30.315944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162da15-fae7-4c62-a2ae-c2ccbe974cda None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:30.315944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162da15-fae7-4c62-a2ae-c2ccbe974cda None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:30.317320 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 743/1476] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:48:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:48:30.323222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e616c-b7e2-4411-988e-5d571bc958e9 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:30.323627 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e616c-b7e2-4411-988e-5d571bc958e9 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:30.323813 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e616c-b7e2-4411-988e-5d571bc958e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:30.390056 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-299e616c-b7e2-4411-988e-5d571bc958e9 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:48:30.391269 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-299e616c-b7e2-4411-988e-5d571bc958e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:30.488098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 734/1477] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:48:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:48:30.767169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a162da15-fae7-4c62-a2ae-c2ccbe974cda None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:48:30.768751 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a162da15-fae7-4c62-a2ae-c2ccbe974cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:30.893191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a162da15-fae7-4c62-a2ae-c2ccbe974cda None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:48:30.941859 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:48:30.986749 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3cf8b-062d-4768-9673-c6b78eefd9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:48:30.987362 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3cf8b-062d-4768-9673-c6b78eefd9bb admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:30.987643 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3cf8b-062d-4768-9673-c6b78eefd9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:30.987915 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3cf8b-062d-4768-9673-c6b78eefd9bb admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:30.988574 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3cf8b-062d-4768-9673-c6b78eefd9bb admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:48:30.990328 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: WARNING py.warnings [None req-4ea3cf8b-062d-4768-9673-c6b78eefd9bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 00:48:30.990328 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: warnings.warn(msg) Aug 26 00:48:30.990328 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: Aug 26 00:48:30.991477 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3cf8b-062d-4768-9673-c6b78eefd9bb admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:48:31.010915 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 744/1478] 10.222.0.59 () {56 vars in 1108 bytes} [Fri Aug 26 00:48:30 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:48:32.941422 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6959e2-8777-4b82-a015-a2f8a9e31c31 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:32.942014 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6959e2-8777-4b82-a015-a2f8a9e31c31 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:32.942014 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6959e2-8777-4b82-a015-a2f8a9e31c31 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:32.943315 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 735/1479] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:48:32.948680 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ca435-422b-4c73-8a2c-7ab4f2fb127b None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:32.949081 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ca435-422b-4c73-8a2c-7ab4f2fb127b None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:32.949235 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ca435-422b-4c73-8a2c-7ab4f2fb127b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:33.015540 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-7a2ca435-422b-4c73-8a2c-7ab4f2fb127b None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:48:33.016768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-7a2ca435-422b-4c73-8a2c-7ab4f2fb127b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:33.116150 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 745/1480] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:48:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:48:33.413074 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6959e2-8777-4b82-a015-a2f8a9e31c31 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:48:33.414563 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-bc6959e2-8777-4b82-a015-a2f8a9e31c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:33.424195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:48:33.558535 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6959e2-8777-4b82-a015-a2f8a9e31c31 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:48:33.636792 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:48:33.668988 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b476a1e-f3a9-4f53-9513-3f3dec737b44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:48:33.669525 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b476a1e-f3a9-4f53-9513-3f3dec737b44 service nova] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:33.669806 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b476a1e-f3a9-4f53-9513-3f3dec737b44 service nova] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:33.670078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b476a1e-f3a9-4f53-9513-3f3dec737b44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:33.671670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-6b476a1e-f3a9-4f53-9513-3f3dec737b44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:33.761368 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b476a1e-f3a9-4f53-9513-3f3dec737b44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:48:33.762952 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b476a1e-f3a9-4f53-9513-3f3dec737b44 service nova] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:48:33.780315 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 736/1481] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:48:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:48:35.854907 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d125a8b-16ec-4f96-a336-5fd135c5f24a None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:35.855464 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d125a8b-16ec-4f96-a336-5fd135c5f24a None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:35.855464 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d125a8b-16ec-4f96-a336-5fd135c5f24a None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:35.856848 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 746/1482] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:48:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:48:35.862198 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b4143-34a7-4cdb-813e-0273cb5a5575 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:35.866202 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b4143-34a7-4cdb-813e-0273cb5a5575 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:35.866202 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b4143-34a7-4cdb-813e-0273cb5a5575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:35.928309 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-074b4143-34a7-4cdb-813e-0273cb5a5575 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:48:35.929549 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-074b4143-34a7-4cdb-813e-0273cb5a5575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:35.975804 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:48:36.026392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 737/1483] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:48:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:48:36.314268 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d125a8b-16ec-4f96-a336-5fd135c5f24a None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:48:36.315806 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-7d125a8b-16ec-4f96-a336-5fd135c5f24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:36.339514 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:48:36.421271 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d125a8b-16ec-4f96-a336-5fd135c5f24a None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:48:36.493939 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b81cad-66b6-47a4-af94-248e942d5e95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:48:36.494551 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b81cad-66b6-47a4-af94-248e942d5e95 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:36.494830 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b81cad-66b6-47a4-af94-248e942d5e95 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:36.495102 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b81cad-66b6-47a4-af94-248e942d5e95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:36.496725 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-d2b81cad-66b6-47a4-af94-248e942d5e95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:36.569381 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:48:36.609188 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b81cad-66b6-47a4-af94-248e942d5e95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:48:36.611225 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b81cad-66b6-47a4-af94-248e942d5e95 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:48:36.629378 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 747/1484] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:48:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:48:38.606108 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75387c6-5340-4964-bb29-52ce580890d7 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:38.606638 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75387c6-5340-4964-bb29-52ce580890d7 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:38.606756 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75387c6-5340-4964-bb29-52ce580890d7 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:38.608042 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 738/1485] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:48:38.613195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e65c9-df68-41fa-a6ad-47eec5d16884 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:38.613614 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e65c9-df68-41fa-a6ad-47eec5d16884 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:38.613751 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e65c9-df68-41fa-a6ad-47eec5d16884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:38.680931 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-bb8e65c9-df68-41fa-a6ad-47eec5d16884 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:48:38.682147 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-bb8e65c9-df68-41fa-a6ad-47eec5d16884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:38.779050 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 748/1486] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:48:39.068766 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75387c6-5340-4964-bb29-52ce580890d7 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:48:39.070494 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-a75387c6-5340-4964-bb29-52ce580890d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:39.174533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75387c6-5340-4964-bb29-52ce580890d7 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:48:39.245600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:48:39.247387 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6f87e-75c0-453f-aef1-ae8aa3517417 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:48:39.247997 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6f87e-75c0-453f-aef1-ae8aa3517417 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:39.248280 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6f87e-75c0-453f-aef1-ae8aa3517417 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:39.248590 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6f87e-75c0-453f-aef1-ae8aa3517417 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:39.250212 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-12e6f87e-75c0-453f-aef1-ae8aa3517417 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:39.358220 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e6f87e-75c0-453f-aef1-ae8aa3517417 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:48:39.359580 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e6f87e-75c0-453f-aef1-ae8aa3517417 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:48:39.377731 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 739/1487] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:48:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:48:41.438905 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c4643-f20a-4573-9f9e-44fbbef4110a None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:41.439347 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c4643-f20a-4573-9f9e-44fbbef4110a None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:41.439456 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c4643-f20a-4573-9f9e-44fbbef4110a None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:41.440841 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 749/1488] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:48:41.446169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98bbeb-8687-4fec-a187-395606147c61 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:41.446574 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98bbeb-8687-4fec-a187-395606147c61 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:41.446726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98bbeb-8687-4fec-a187-395606147c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:41.512021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-ad98bbeb-8687-4fec-a187-395606147c61 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:48:41.513256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-ad98bbeb-8687-4fec-a187-395606147c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:41.566245 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:48:41.611233 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 740/1489] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:48:41.904671 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c4643-f20a-4573-9f9e-44fbbef4110a None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:48:41.906238 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-e07c4643-f20a-4573-9f9e-44fbbef4110a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:41.942415 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:48:42.009848 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c4643-f20a-4573-9f9e-44fbbef4110a None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:48:42.082370 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca7dd0-c130-44b5-acc5-4f717a48cda3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:48:42.082975 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca7dd0-c130-44b5-acc5-4f717a48cda3 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:42.083251 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca7dd0-c130-44b5-acc5-4f717a48cda3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:42.083517 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca7dd0-c130-44b5-acc5-4f717a48cda3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:42.085127 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-edca7dd0-c130-44b5-acc5-4f717a48cda3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:42.164364 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:48:42.193271 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca7dd0-c130-44b5-acc5-4f717a48cda3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:48:42.194723 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca7dd0-c130-44b5-acc5-4f717a48cda3 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:48:42.213795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 750/1490] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:48:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:48:44.574259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ac333-691e-4957-8d55-0970f3e6b5a0 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:44.574822 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ac333-691e-4957-8d55-0970f3e6b5a0 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:44.574822 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ac333-691e-4957-8d55-0970f3e6b5a0 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:44.576157 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 741/1491] 10.222.0.59 () {54 vars in 927 bytes} [Fri Aug 26 00:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:48:44.581428 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ace02-73b7-4870-b09c-b2da259795e4 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:44.581860 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ace02-73b7-4870-b09c-b2da259795e4 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:44.581972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ace02-73b7-4870-b09c-b2da259795e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:44.648973 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-be2ace02-73b7-4870-b09c-b2da259795e4 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:48:44.650178 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-be2ace02-73b7-4870-b09c-b2da259795e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:44.746886 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 751/1492] 10.222.0.59 () {58 vars in 1024 bytes} [Fri Aug 26 00:48:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:48:45.035013 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ac333-691e-4957-8d55-0970f3e6b5a0 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:48:45.036656 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-3e0ac333-691e-4957-8d55-0970f3e6b5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:45.141856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ac333-691e-4957-8d55-0970f3e6b5a0 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:48:45.218564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88af1b-67db-47ee-ab0a-2e0fff1347c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:48:45.219168 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88af1b-67db-47ee-ab0a-2e0fff1347c2 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:48:45.219449 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88af1b-67db-47ee-ab0a-2e0fff1347c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:48:45.219722 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88af1b-67db-47ee-ab0a-2e0fff1347c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:48:45.221353 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-fc88af1b-67db-47ee-ab0a-2e0fff1347c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:48:45.230951 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:48:45.328646 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc88af1b-67db-47ee-ab0a-2e0fff1347c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:48:45.330116 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc88af1b-67db-47ee-ab0a-2e0fff1347c2 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:48:45.348182 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 742/1493] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:48:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:48:54.744952 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=82 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:48:55.345977 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=50 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:50:08.092522 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0ea43-7cd0-4623-9655-27ec5f6972d7 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:50:08.093119 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0ea43-7cd0-4623-9655-27ec5f6972d7 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:50:08.093119 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0ea43-7cd0-4623-9655-27ec5f6972d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:50:08.167554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-16c0ea43-7cd0-4623-9655-27ec5f6972d7 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:50:08.168782 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-16c0ea43-7cd0-4623-9655-27ec5f6972d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:50:08.263948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 752/1494] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:50:08.273203 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f9cfd-2c6f-48a9-8707-7577fa187671 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:50:08.273708 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f9cfd-2c6f-48a9-8707-7577fa187671 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:50:08.273820 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f9cfd-2c6f-48a9-8707-7577fa187671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:50:08.349972 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-ee8f9cfd-2c6f-48a9-8707-7577fa187671 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:50:08.351105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-ee8f9cfd-2c6f-48a9-8707-7577fa187671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:50:08.448177 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 743/1495] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:50:08.452897 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0ea43-7cd0-4623-9655-27ec5f6972d7 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:50:08.454087 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-16c0ea43-7cd0-4623-9655-27ec5f6972d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:50:08.465799 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0ea43-7cd0-4623-9655-27ec5f6972d7 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:50:08.525468 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:50:08.559424 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baedb26-286c-45d3-9b46-443d4d16209e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:50:08.560029 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baedb26-286c-45d3-9b46-443d4d16209e admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:50:08.560308 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baedb26-286c-45d3-9b46-443d4d16209e admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:50:08.560597 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baedb26-286c-45d3-9b46-443d4d16209e admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:50:08.561302 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baedb26-286c-45d3-9b46-443d4d16209e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:50:08.561621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baedb26-286c-45d3-9b46-443d4d16209e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:50:08.562944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: WARNING py.warnings [None req-4baedb26-286c-45d3-9b46-443d4d16209e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 00:50:08.562944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: warnings.warn(msg) Aug 26 00:50:08.562944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: Aug 26 00:50:08.564068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baedb26-286c-45d3-9b46-443d4d16209e admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:50:08.575311 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 753/1496] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:50:08.584344 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f9cfd-2c6f-48a9-8707-7577fa187671 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:50:08.593502 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f9cfd-2c6f-48a9-8707-7577fa187671 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:50:08.653126 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:50:08.686576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae511e61-6dfe-44cc-a4b1-1dc953d5d29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:50:08.687181 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae511e61-6dfe-44cc-a4b1-1dc953d5d29e admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:50:08.687455 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae511e61-6dfe-44cc-a4b1-1dc953d5d29e admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:50:08.687720 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae511e61-6dfe-44cc-a4b1-1dc953d5d29e admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:50:08.688340 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae511e61-6dfe-44cc-a4b1-1dc953d5d29e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:50:08.690568 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: WARNING py.warnings [None req-ae511e61-6dfe-44cc-a4b1-1dc953d5d29e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 00:50:08.690568 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: warnings.warn(msg) Aug 26 00:50:08.690568 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: Aug 26 00:50:08.692067 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae511e61-6dfe-44cc-a4b1-1dc953d5d29e admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:50:08.805216 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 744/1497] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:50:08 2022] POST /identity/v3/projects => generated 379 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:50:08.809752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baedb26-286c-45d3-9b46-443d4d16209e admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:50:08.811289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-4baedb26-286c-45d3-9b46-443d4d16209e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:50:08.845027 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baedb26-286c-45d3-9b46-443d4d16209e admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:50:08.943638 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf4f65d-614e-4acb-850e-ad3b0badd408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:50:08.944230 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf4f65d-614e-4acb-850e-ad3b0badd408 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:50:08.944594 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf4f65d-614e-4acb-850e-ad3b0badd408 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:50:08.944874 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf4f65d-614e-4acb-850e-ad3b0badd408 admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:50:08.945619 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf4f65d-614e-4acb-850e-ad3b0badd408 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:50:08.946646 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: WARNING py.warnings [None req-3bf4f65d-614e-4acb-850e-ad3b0badd408 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 00:50:08.946646 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: warnings.warn(msg) Aug 26 00:50:08.946646 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: Aug 26 00:50:08.947860 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf4f65d-614e-4acb-850e-ad3b0badd408 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:50:09.078768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 754/1498] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:50:08 2022] POST /identity/v3/users => generated 290 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:50:09.083058 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae511e61-6dfe-44cc-a4b1-1dc953d5d29e admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:50:09.084596 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-ae511e61-6dfe-44cc-a4b1-1dc953d5d29e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:50:09.145863 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae511e61-6dfe-44cc-a4b1-1dc953d5d29e admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:50:09.231266 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:50:09.244419 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be05ef-3fde-4ed3-b7b8-5964a3d62b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:50:09.247681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be05ef-3fde-4ed3-b7b8-5964a3d62b3d admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:50:09.247681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be05ef-3fde-4ed3-b7b8-5964a3d62b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:50:09.247681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be05ef-3fde-4ed3-b7b8-5964a3d62b3d admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:50:09.247681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be05ef-3fde-4ed3-b7b8-5964a3d62b3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:50:09.247681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: WARNING py.warnings [None req-d6be05ef-3fde-4ed3-b7b8-5964a3d62b3d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 00:50:09.247681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: warnings.warn(msg) Aug 26 00:50:09.247681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: Aug 26 00:50:09.248353 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be05ef-3fde-4ed3-b7b8-5964a3d62b3d admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:50:09.260759 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 745/1499] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:50:09.264963 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf4f65d-614e-4acb-850e-ad3b0badd408 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:50:09.266647 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-3bf4f65d-614e-4acb-850e-ad3b0badd408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:50:09.278453 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf4f65d-614e-4acb-850e-ad3b0badd408 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:50:09.303963 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:50:09.372776 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf64190e-28a0-44da-9374-a1a0a15e9bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:50:09.373379 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64190e-28a0-44da-9374-a1a0a15e9bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:50:09.373657 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64190e-28a0-44da-9374-a1a0a15e9bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:50:09.373928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64190e-28a0-44da-9374-a1a0a15e9bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:50:09.374571 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf64190e-28a0-44da-9374-a1a0a15e9bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:50:09.375615 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: WARNING py.warnings [None req-bf64190e-28a0-44da-9374-a1a0a15e9bd6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 00:50:09.375615 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: warnings.warn(msg) Aug 26 00:50:09.375615 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: Aug 26 00:50:09.376528 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf64190e-28a0-44da-9374-a1a0a15e9bd6 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:50:09.389023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 755/1500] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:50:09.393810 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be05ef-3fde-4ed3-b7b8-5964a3d62b3d admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:50:09.395354 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-d6be05ef-3fde-4ed3-b7b8-5964a3d62b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:50:09.429001 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be05ef-3fde-4ed3-b7b8-5964a3d62b3d admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:50:09.524753 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b91e7-c820-4807-aa06-105bf5d49035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:50:09.525333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b91e7-c820-4807-aa06-105bf5d49035 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:50:09.525612 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b91e7-c820-4807-aa06-105bf5d49035 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:50:09.525884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b91e7-c820-4807-aa06-105bf5d49035 admin admin] PATH_INFO: `/v3/projects/c1735c3bf2434a578840e1eeaf60362f/users/5afbda65572e48c7b388645a97967858/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:50:09.547835 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b91e7-c820-4807-aa06-105bf5d49035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1735c3bf2434a578840e1eeaf60362f, user_id=5afbda65572e48c7b388645a97967858, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:50:09.548898 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: WARNING py.warnings [None req-651b91e7-c820-4807-aa06-105bf5d49035 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 00:50:09.548898 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: warnings.warn(msg) Aug 26 00:50:09.548898 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: Aug 26 00:50:09.550923 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b91e7-c820-4807-aa06-105bf5d49035 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:50:09.639928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 746/1501] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:50:09 2022] PUT /identity/v3/projects/c1735c3bf2434a578840e1eeaf60362f/users/5afbda65572e48c7b388645a97967858/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:50:09.652943 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e177b9-d0ef-4a66-8f98-4be0eab942af None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:50:09.653237 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e177b9-d0ef-4a66-8f98-4be0eab942af None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:50:09.653523 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e177b9-d0ef-4a66-8f98-4be0eab942af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:50:09.711260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-73e177b9-d0ef-4a66-8f98-4be0eab942af None None] MFA Rules not processed for user `5afbda65572e48c7b388645a97967858`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:50:09.712498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-73e177b9-d0ef-4a66-8f98-4be0eab942af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:50:09.789114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 756/1502] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:50:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:50:09.802129 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3178c4-6940-4694-91e1-2e68dd21c67b None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:50:09.802419 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3178c4-6940-4694-91e1-2e68dd21c67b None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:50:09.802692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3178c4-6940-4694-91e1-2e68dd21c67b None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:50:09.804022 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 747/1503] 10.222.0.59 () {54 vars in 964 bytes} [Fri Aug 26 00:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:50:09.808888 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fd9a1-58c7-40d7-8582-a4ac4697ecce None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:50:09.809178 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fd9a1-58c7-40d7-8582-a4ac4697ecce None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:50:09.809457 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fd9a1-58c7-40d7-8582-a4ac4697ecce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:50:09.841159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:50:09.885751 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-7a3fd9a1-58c7-40d7-8582-a4ac4697ecce None None] MFA Rules not processed for user `c88202614775418ba94f478f03431a9b`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:50:09.887128 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-7a3fd9a1-58c7-40d7-8582-a4ac4697ecce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:50:09.962771 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 757/1504] 10.222.0.59 () {58 vars in 1061 bytes} [Fri Aug 26 00:50:09 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:50:09.970764 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3178c4-6940-4694-91e1-2e68dd21c67b None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:50:09.972881 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-3e3178c4-6940-4694-91e1-2e68dd21c67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:50:10.021430 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:50:10.074751 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3178c4-6940-4694-91e1-2e68dd21c67b None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:50:10.146955 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d79f24-0c47-4316-9e22-40ad2b2966b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:50:10.147553 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d79f24-0c47-4316-9e22-40ad2b2966b7 service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:50:10.147831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d79f24-0c47-4316-9e22-40ad2b2966b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:50:10.148101 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d79f24-0c47-4316-9e22-40ad2b2966b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:50:10.149715 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-06d79f24-0c47-4316-9e22-40ad2b2966b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:50:10.245954 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d79f24-0c47-4316-9e22-40ad2b2966b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:50:10.247379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d79f24-0c47-4316-9e22-40ad2b2966b7 service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:50:10.250833 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:50:10.265443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 748/1505] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:50:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:50:10.310804 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd8ec3-57a4-4750-abc3-e520e8034fe4 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:50:10.311217 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd8ec3-57a4-4750-abc3-e520e8034fe4 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:50:10.311341 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd8ec3-57a4-4750-abc3-e520e8034fe4 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:50:10.312640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 758/1506] 10.222.0.59 () {52 vars in 830 bytes} [Fri Aug 26 00:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:50:10.316599 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d79f24-0c47-4316-9e22-40ad2b2966b7 service cinder] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:50:10.317817 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-06d79f24-0c47-4316-9e22-40ad2b2966b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:50:10.416244 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d79f24-0c47-4316-9e22-40ad2b2966b7 service cinder] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:50:10.509074 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be710d72-99bb-473b-a207-033163fb42a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:50:10.509710 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be710d72-99bb-473b-a207-033163fb42a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:50:10.509991 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be710d72-99bb-473b-a207-033163fb42a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:50:10.510266 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be710d72-99bb-473b-a207-033163fb42a5 admin admin] PATH_INFO: `/v3/users/5afbda65572e48c7b388645a97967858` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:50:10.522213 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be710d72-99bb-473b-a207-033163fb42a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5afbda65572e48c7b388645a97967858)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:50:10.523243 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: WARNING py.warnings [None req-be710d72-99bb-473b-a207-033163fb42a5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 00:50:10.523243 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: warnings.warn(msg) Aug 26 00:50:10.523243 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: Aug 26 00:50:10.524455 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be710d72-99bb-473b-a207-033163fb42a5 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:50:10.534731 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:50:10.753586 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-be710d72-99bb-473b-a207-033163fb42a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5afbda65572e48c7b388645a97967858'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:50:10.775892 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-be710d72-99bb-473b-a207-033163fb42a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5afbda65572e48c7b388645a97967858'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:50:10.872516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-be710d72-99bb-473b-a207-033163fb42a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5afbda65572e48c7b388645a97967858'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:50:10.872957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-be710d72-99bb-473b-a207-033163fb42a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5afbda65572e48c7b388645a97967858'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:50:10.873482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-be710d72-99bb-473b-a207-033163fb42a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5afbda65572e48c7b388645a97967858'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:50:10.895101 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 749/1507] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:50:10 2022] DELETE /identity/v3/users/5afbda65572e48c7b388645a97967858 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:50:10.899491 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005f8b4-dbe0-4afa-8f1b-9b910daf1fb7 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:50:10.899848 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005f8b4-dbe0-4afa-8f1b-9b910daf1fb7 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:50:10.900188 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005f8b4-dbe0-4afa-8f1b-9b910daf1fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:50:10.960860 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-9005f8b4-dbe0-4afa-8f1b-9b910daf1fb7 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:50:10.962089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-9005f8b4-dbe0-4afa-8f1b-9b910daf1fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:50:11.037352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:50:11.049765 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 759/1508] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:50:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:50:11.060095 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be710d72-99bb-473b-a207-033163fb42a5 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:50:11.061339 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-be710d72-99bb-473b-a207-033163fb42a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:50:11.157938 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be710d72-99bb-473b-a207-033163fb42a5 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:50:11.210954 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:50:11.228689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335ff3f-5f2a-4e37-98cc-18b286451953 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:50:11.229292 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335ff3f-5f2a-4e37-98cc-18b286451953 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:50:11.229569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335ff3f-5f2a-4e37-98cc-18b286451953 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:50:11.229836 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335ff3f-5f2a-4e37-98cc-18b286451953 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:50:11.231411 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-1335ff3f-5f2a-4e37-98cc-18b286451953 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:50:11.336509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335ff3f-5f2a-4e37-98cc-18b286451953 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:50:11.337944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335ff3f-5f2a-4e37-98cc-18b286451953 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:50:11.355626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 750/1509] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:50:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:50:11.844450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005f8b4-dbe0-4afa-8f1b-9b910daf1fb7 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:50:11.845765 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-9005f8b4-dbe0-4afa-8f1b-9b910daf1fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:50:11.858344 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005f8b4-dbe0-4afa-8f1b-9b910daf1fb7 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:50:11.954573 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec967b7b-e2df-4b3d-be21-8448cbfd6097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:50:11.955237 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec967b7b-e2df-4b3d-be21-8448cbfd6097 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:50:11.955516 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec967b7b-e2df-4b3d-be21-8448cbfd6097 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:50:11.955789 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec967b7b-e2df-4b3d-be21-8448cbfd6097 admin admin] PATH_INFO: `/v3/projects/c1735c3bf2434a578840e1eeaf60362f` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:50:11.965654 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec967b7b-e2df-4b3d-be21-8448cbfd6097 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1735c3bf2434a578840e1eeaf60362f)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:50:11.966684 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: WARNING py.warnings [None req-ec967b7b-e2df-4b3d-be21-8448cbfd6097 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 00:50:11.966684 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: warnings.warn(msg) Aug 26 00:50:11.966684 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: Aug 26 00:50:11.967893 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec967b7b-e2df-4b3d-be21-8448cbfd6097 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:50:12.110833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ec967b7b-e2df-4b3d-be21-8448cbfd6097 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1735c3bf2434a578840e1eeaf60362f', 'request_id': 'req-ec967b7b-e2df-4b3d-be21-8448cbfd6097'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:50:12.181189 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ec967b7b-e2df-4b3d-be21-8448cbfd6097 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1735c3bf2434a578840e1eeaf60362f', 'request_id': 'req-ec967b7b-e2df-4b3d-be21-8448cbfd6097'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:50:12.187641 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ec967b7b-e2df-4b3d-be21-8448cbfd6097 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1735c3bf2434a578840e1eeaf60362f', 'request_id': 'req-ec967b7b-e2df-4b3d-be21-8448cbfd6097'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:50:12.218734 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ec967b7b-e2df-4b3d-be21-8448cbfd6097 admin admin] The token cache is being invalidate because project c1735c3bf2434a578840e1eeaf60362f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:50:12.219027 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ec967b7b-e2df-4b3d-be21-8448cbfd6097 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:50:12.219561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ec967b7b-e2df-4b3d-be21-8448cbfd6097 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:50:12.220917 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 760/1510] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:50:11 2022] DELETE /identity/v3/projects/c1735c3bf2434a578840e1eeaf60362f => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:50:21.358939 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=62 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:50:22.216699 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:51:41.423227 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c31a8-4ca6-4cce-ab20-8037d1544203 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:51:41.423847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c31a8-4ca6-4cce-ab20-8037d1544203 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:51:41.423847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c31a8-4ca6-4cce-ab20-8037d1544203 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:51:41.425158 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 00:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:51:41.430018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef965c-d3ae-48a8-8bbf-4cfd4aa53c01 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:51:41.430470 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef965c-d3ae-48a8-8bbf-4cfd4aa53c01 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:51:41.430578 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef965c-d3ae-48a8-8bbf-4cfd4aa53c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:51:41.511319 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-44ef965c-d3ae-48a8-8bbf-4cfd4aa53c01 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:51:41.512550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-44ef965c-d3ae-48a8-8bbf-4cfd4aa53c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:51:41.616872 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 00:51:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:51:41.774184 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c31a8-4ca6-4cce-ab20-8037d1544203 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:51:41.776445 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-d81c31a8-4ca6-4cce-ab20-8037d1544203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:51:41.893178 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c31a8-4ca6-4cce-ab20-8037d1544203 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:51:41.898549 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b237c5c-81ff-4339-9b27-b6e8512be370 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:51:41.899181 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b237c5c-81ff-4339-9b27-b6e8512be370 service nova] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:51:41.899458 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b237c5c-81ff-4339-9b27-b6e8512be370 service nova] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:51:41.899726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b237c5c-81ff-4339-9b27-b6e8512be370 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:51:41.907500 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b237c5c-81ff-4339-9b27-b6e8512be370 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:51:41.909398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b237c5c-81ff-4339-9b27-b6e8512be370 service nova] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:51:41.918486 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 752/1513] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:51:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:51:42.748370 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a8679-f215-4dda-a0f2-2c02597ec67d None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:51:42.748931 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a8679-f215-4dda-a0f2-2c02597ec67d None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:51:42.748931 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a8679-f215-4dda-a0f2-2c02597ec67d None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:51:42.750230 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 00:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:51:42.754997 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831489e-bf47-4471-bbd7-3dda47f1b17b None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:51:42.755402 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831489e-bf47-4471-bbd7-3dda47f1b17b None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:51:42.755555 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831489e-bf47-4471-bbd7-3dda47f1b17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:51:42.775631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-8831489e-bf47-4471-bbd7-3dda47f1b17b None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:51:42.776911 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-8831489e-bf47-4471-bbd7-3dda47f1b17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:51:42.785795 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 00:51:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:51:42.942478 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a8679-f215-4dda-a0f2-2c02597ec67d None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:51:42.957874 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a8679-f215-4dda-a0f2-2c02597ec67d None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:51:42.963228 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc80670-62a0-4897-b5a1-b3b89545b034 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:51:42.963914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc80670-62a0-4897-b5a1-b3b89545b034 service nova] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:51:42.964195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc80670-62a0-4897-b5a1-b3b89545b034 service nova] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:51:42.964463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc80670-62a0-4897-b5a1-b3b89545b034 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:51:42.972642 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc80670-62a0-4897-b5a1-b3b89545b034 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:51:42.974319 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc80670-62a0-4897-b5a1-b3b89545b034 service nova] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:51:42.983666 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 763/1516] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:51:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:51:43.258795 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0bd08-f7ca-4346-af0a-69f8512cd7e4 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:51:43.259080 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0bd08-f7ca-4346-af0a-69f8512cd7e4 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:51:43.259411 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0bd08-f7ca-4346-af0a-69f8512cd7e4 None None] PATH_INFO: `/v3` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:51:43.260745 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 754/1517] 10.222.0.91 () {54 vars in 925 bytes} [Fri Aug 26 00:51:43 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:51:43.276167 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b96ac-d939-4575-ad16-96ddcb4d8aa0 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:51:43.276394 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b96ac-d939-4575-ad16-96ddcb4d8aa0 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:51:43.276736 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b96ac-d939-4575-ad16-96ddcb4d8aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:51:43.337611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-942b96ac-d939-4575-ad16-96ddcb4d8aa0 None None] MFA Rules not processed for user `2876f4182f0949039b862f053533fe85`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:51:43.338823 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-942b96ac-d939-4575-ad16-96ddcb4d8aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:51:43.355413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:51:43.409997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 764/1518] 10.222.0.91 () {58 vars in 1015 bytes} [Fri Aug 26 00:51:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:51:52.732051 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab3315-df4b-49e2-b4c6-a5e9240f5a95 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:51:52.732630 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab3315-df4b-49e2-b4c6-a5e9240f5a95 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:51:52.732630 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab3315-df4b-49e2-b4c6-a5e9240f5a95 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:51:52.733875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 755/1519] 10.222.0.59 () {54 vars in 961 bytes} [Fri Aug 26 00:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:51:52.739188 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf390d-2827-4f1b-b524-b3e0fcba9b41 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:51:52.743349 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf390d-2827-4f1b-b524-b3e0fcba9b41 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:51:52.743349 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf390d-2827-4f1b-b524-b3e0fcba9b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:51:52.759471 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-92bf390d-2827-4f1b-b524-b3e0fcba9b41 None None] MFA Rules not processed for user `2876f4182f0949039b862f053533fe85`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:51:52.760681 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-92bf390d-2827-4f1b-b524-b3e0fcba9b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:51:52.769328 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 765/1520] 10.222.0.59 () {58 vars in 1058 bytes} [Fri Aug 26 00:51:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:51:52.772074 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=56 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:51:52.777037 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab3315-df4b-49e2-b4c6-a5e9240f5a95 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:51:52.791026 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab3315-df4b-49e2-b4c6-a5e9240f5a95 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:51:52.796014 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708430eb-c5cf-42f2-b434-f59da3898adf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2876f4182f0949039b862f053533fe85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:51:52.796705 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708430eb-c5cf-42f2-b434-f59da3898adf service placement] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:51:52.800744 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708430eb-c5cf-42f2-b434-f59da3898adf service placement] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:51:52.800744 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708430eb-c5cf-42f2-b434-f59da3898adf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:51:52.806115 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708430eb-c5cf-42f2-b434-f59da3898adf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:51:52.807697 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708430eb-c5cf-42f2-b434-f59da3898adf service placement] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:51:52.817023 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 756/1521] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:51:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:51:54.337408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51ebca-babb-40d9-8373-1ecd7b51e96d None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:51:54.337966 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51ebca-babb-40d9-8373-1ecd7b51e96d None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:51:54.337966 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51ebca-babb-40d9-8373-1ecd7b51e96d None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:51:54.339285 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 00:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:51:54.344020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22742575-6ddd-440f-8d0e-df0ccc9cdc58 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:51:54.344497 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22742575-6ddd-440f-8d0e-df0ccc9cdc58 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:51:54.344619 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22742575-6ddd-440f-8d0e-df0ccc9cdc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:51:54.363552 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-22742575-6ddd-440f-8d0e-df0ccc9cdc58 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:51:54.364758 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-22742575-6ddd-440f-8d0e-df0ccc9cdc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:51:54.372415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 00:51:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:51:54.528504 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51ebca-babb-40d9-8373-1ecd7b51e96d None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:51:54.546092 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51ebca-babb-40d9-8373-1ecd7b51e96d None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:51:54.551725 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30ea93-3516-41e4-80a7-edea7c237a57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:51:54.552360 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30ea93-3516-41e4-80a7-edea7c237a57 service nova] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:51:54.552671 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30ea93-3516-41e4-80a7-edea7c237a57 service nova] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:51:54.552948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30ea93-3516-41e4-80a7-edea7c237a57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:51:54.562163 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce30ea93-3516-41e4-80a7-edea7c237a57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:51:54.563699 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce30ea93-3516-41e4-80a7-edea7c237a57 service nova] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:51:54.574028 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 767/1524] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:51:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:51:55.402560 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f122db4-893e-4b20-982a-5344fbe65cab None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:51:55.403134 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f122db4-893e-4b20-982a-5344fbe65cab None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:51:55.403399 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f122db4-893e-4b20-982a-5344fbe65cab None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:51:55.405503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 00:51:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:51:55.412547 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a82c00-4471-47b6-ba7c-cf2872e59144 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:51:55.413012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a82c00-4471-47b6-ba7c-cf2872e59144 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:51:55.413125 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a82c00-4471-47b6-ba7c-cf2872e59144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:51:55.433330 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-87a82c00-4471-47b6-ba7c-cf2872e59144 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:51:55.434553 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-87a82c00-4471-47b6-ba7c-cf2872e59144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:51:55.442784 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 00:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:51:55.596726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f122db4-893e-4b20-982a-5344fbe65cab None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:51:55.611361 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f122db4-893e-4b20-982a-5344fbe65cab None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:51:55.617290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba47a9a-fd76-41aa-8023-c3818cb3f163 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:51:55.617880 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba47a9a-fd76-41aa-8023-c3818cb3f163 service nova] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:51:55.618179 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba47a9a-fd76-41aa-8023-c3818cb3f163 service nova] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:51:55.618432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba47a9a-fd76-41aa-8023-c3818cb3f163 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:51:55.630268 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba47a9a-fd76-41aa-8023-c3818cb3f163 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:51:55.633079 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba47a9a-fd76-41aa-8023-c3818cb3f163 service nova] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:51:55.649778 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 759/1527] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:51:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:51:59.650058 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfeeb6-9212-4b0f-aad4-8019f6eb3fa1 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:51:59.650630 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfeeb6-9212-4b0f-aad4-8019f6eb3fa1 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:51:59.650630 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfeeb6-9212-4b0f-aad4-8019f6eb3fa1 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:51:59.651955 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 00:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:51:59.657482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a78a0-6668-42d4-abe8-ca5032116837 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:51:59.657900 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a78a0-6668-42d4-abe8-ca5032116837 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:51:59.658052 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a78a0-6668-42d4-abe8-ca5032116837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:51:59.679593 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-6a2a78a0-6668-42d4-abe8-ca5032116837 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:51:59.680825 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-6a2a78a0-6668-42d4-abe8-ca5032116837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:51:59.688430 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 00:51:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:51:59.982298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbfeeb6-9212-4b0f-aad4-8019f6eb3fa1 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:51:59.984008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-1fbfeeb6-9212-4b0f-aad4-8019f6eb3fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:52:00.062176 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbfeeb6-9212-4b0f-aad4-8019f6eb3fa1 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:52:00.065671 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710231ff-1539-4863-9d4b-ccf53a839782 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:52:00.066277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710231ff-1539-4863-9d4b-ccf53a839782 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:52:00.066556 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710231ff-1539-4863-9d4b-ccf53a839782 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:52:00.066825 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710231ff-1539-4863-9d4b-ccf53a839782 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:52:00.075594 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710231ff-1539-4863-9d4b-ccf53a839782 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:52:00.077050 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710231ff-1539-4863-9d4b-ccf53a839782 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:52:00.086334 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 770/1530] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:51:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:52:01.988206 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d604e-a1a2-49ca-9c7e-82a0c9cecc68 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:52:01.988764 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d604e-a1a2-49ca-9c7e-82a0c9cecc68 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:52:01.988764 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d604e-a1a2-49ca-9c7e-82a0c9cecc68 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:52:01.990081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 00:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:52:01.995399 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d7bbe-7b55-45ba-85b2-547ca47911e3 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:52:01.995835 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d7bbe-7b55-45ba-85b2-547ca47911e3 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:52:01.995963 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d7bbe-7b55-45ba-85b2-547ca47911e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:52:02.015018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-d69d7bbe-7b55-45ba-85b2-547ca47911e3 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:52:02.016219 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-d69d7bbe-7b55-45ba-85b2-547ca47911e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:52:02.024567 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 00:52:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:52:02.308216 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d604e-a1a2-49ca-9c7e-82a0c9cecc68 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:52:02.319702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d604e-a1a2-49ca-9c7e-82a0c9cecc68 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:52:02.323424 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6384dc-d3db-4ee3-b0a7-a33736998a01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:52:02.324019 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6384dc-d3db-4ee3-b0a7-a33736998a01 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:52:02.324299 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6384dc-d3db-4ee3-b0a7-a33736998a01 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:52:02.324605 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6384dc-d3db-4ee3-b0a7-a33736998a01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:52:02.333371 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6384dc-d3db-4ee3-b0a7-a33736998a01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:52:02.335399 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6384dc-d3db-4ee3-b0a7-a33736998a01 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:52:02.344773 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 762/1533] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:52:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:52:05.680033 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73140054-136e-4dd4-b4dc-31c78bbef6ce None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:52:05.680663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73140054-136e-4dd4-b4dc-31c78bbef6ce None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:52:05.680663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73140054-136e-4dd4-b4dc-31c78bbef6ce None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:52:05.682014 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 00:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:52:05.687206 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852bdb3-3714-4f5e-8152-a064abb9a33f None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:52:05.687634 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852bdb3-3714-4f5e-8152-a064abb9a33f None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:52:05.687634 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852bdb3-3714-4f5e-8152-a064abb9a33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:52:05.707679 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-4852bdb3-3714-4f5e-8152-a064abb9a33f None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:52:05.708916 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-4852bdb3-3714-4f5e-8152-a064abb9a33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:52:05.716863 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 00:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:52:06.271034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73140054-136e-4dd4-b4dc-31c78bbef6ce None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:52:06.272780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-73140054-136e-4dd4-b4dc-31c78bbef6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:52:06.339380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:52:06.353386 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73140054-136e-4dd4-b4dc-31c78bbef6ce None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:52:06.359246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40729808-d6f0-4cd9-8dc5-ac39d9d26c4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:52:06.359246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40729808-d6f0-4cd9-8dc5-ac39d9d26c4d service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:52:06.359246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40729808-d6f0-4cd9-8dc5-ac39d9d26c4d service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:52:06.359246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40729808-d6f0-4cd9-8dc5-ac39d9d26c4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:52:06.367438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40729808-d6f0-4cd9-8dc5-ac39d9d26c4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:52:06.368861 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40729808-d6f0-4cd9-8dc5-ac39d9d26c4d service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:52:06.378088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 773/1536] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:52:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:52:08.307590 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55393c7-8f82-4423-8e87-cb380c56f19d None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:52:08.308203 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55393c7-8f82-4423-8e87-cb380c56f19d None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:52:08.308203 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55393c7-8f82-4423-8e87-cb380c56f19d None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:52:08.309568 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 00:52:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:52:08.314875 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39707591-d997-4ea2-8fac-1aa1b2181a4e None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:52:08.315268 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39707591-d997-4ea2-8fac-1aa1b2181a4e None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:52:08.315428 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39707591-d997-4ea2-8fac-1aa1b2181a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:52:08.334882 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-39707591-d997-4ea2-8fac-1aa1b2181a4e None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:52:08.336709 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-39707591-d997-4ea2-8fac-1aa1b2181a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:52:08.344413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 00:52:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:52:08.886553 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55393c7-8f82-4423-8e87-cb380c56f19d None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:52:08.888295 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-a55393c7-8f82-4423-8e87-cb380c56f19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:52:08.901617 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55393c7-8f82-4423-8e87-cb380c56f19d None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:52:08.905148 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1495a-0354-49ae-a3e7-8069382d25a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:52:08.905725 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1495a-0354-49ae-a3e7-8069382d25a7 service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:52:08.906002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1495a-0354-49ae-a3e7-8069382d25a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:52:08.906267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1495a-0354-49ae-a3e7-8069382d25a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:52:08.916116 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1495a-0354-49ae-a3e7-8069382d25a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:52:08.917572 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1495a-0354-49ae-a3e7-8069382d25a7 service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:52:08.926819 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 765/1539] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:52:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:52:18.329830 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=11 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:52:18.922760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=33 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:33.507541 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a81995-965a-4186-a04f-52522e917e58 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:33.508222 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a81995-965a-4186-a04f-52522e917e58 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:33.508222 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a81995-965a-4186-a04f-52522e917e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:33.532047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-81a81995-965a-4186-a04f-52522e917e58 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:33.533663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-81a81995-965a-4186-a04f-52522e917e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:33.545479 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 775/1540] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:33.555394 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac0c61-c19b-4b9f-99d3-67aabb97621b None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:33.555999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac0c61-c19b-4b9f-99d3-67aabb97621b None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:33.555999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac0c61-c19b-4b9f-99d3-67aabb97621b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:33.583382 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-3eac0c61-c19b-4b9f-99d3-67aabb97621b None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:33.585040 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-3eac0c61-c19b-4b9f-99d3-67aabb97621b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:33.596126 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 766/1541] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:33.601758 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a81995-965a-4186-a04f-52522e917e58 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:33.613005 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a81995-965a-4186-a04f-52522e917e58 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:33.622911 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be405a25-e327-4617-9e57-fc7baf6dc1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:33.623557 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be405a25-e327-4617-9e57-fc7baf6dc1a2 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:33.623842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be405a25-e327-4617-9e57-fc7baf6dc1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:33.624602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be405a25-e327-4617-9e57-fc7baf6dc1a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:33.625425 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be405a25-e327-4617-9e57-fc7baf6dc1a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:53:33.625738 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be405a25-e327-4617-9e57-fc7baf6dc1a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:33.627438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be405a25-e327-4617-9e57-fc7baf6dc1a2 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:33.646116 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 776/1542] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:53:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:53:33.656191 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac0c61-c19b-4b9f-99d3-67aabb97621b None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:33.670735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac0c61-c19b-4b9f-99d3-67aabb97621b None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:33.676324 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ec48c-4ced-492c-9e7d-bb16efb1f9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:33.677287 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ec48c-4ced-492c-9e7d-bb16efb1f9d4 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:33.678760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ec48c-4ced-492c-9e7d-bb16efb1f9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:33.679213 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ec48c-4ced-492c-9e7d-bb16efb1f9d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:33.680037 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ec48c-4ced-492c-9e7d-bb16efb1f9d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:33.682232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ec48c-4ced-492c-9e7d-bb16efb1f9d4 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:33.804409 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd1286-d926-42bc-85ba-4b89eb353eee None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:33.804936 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd1286-d926-42bc-85ba-4b89eb353eee None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:33.805062 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd1286-d926-42bc-85ba-4b89eb353eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:33.809981 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 767/1543] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:53:33 2022] POST /identity/v3/projects => generated 379 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:53:33.814526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ec48c-4ced-492c-9e7d-bb16efb1f9d4 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:33.842181 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-7ebd1286-d926-42bc-85ba-4b89eb353eee None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:33.843434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-7ebd1286-d926-42bc-85ba-4b89eb353eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:33.900488 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ec48c-4ced-492c-9e7d-bb16efb1f9d4 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:33.911165 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392e486-8ca4-4ccf-9cc6-c3383faa493a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:33.911789 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392e486-8ca4-4ccf-9cc6-c3383faa493a admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:33.912896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392e486-8ca4-4ccf-9cc6-c3383faa493a admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:33.913167 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392e486-8ca4-4ccf-9cc6-c3383faa493a admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:33.913806 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f392e486-8ca4-4ccf-9cc6-c3383faa493a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:33.914824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: WARNING py.warnings [None req-f392e486-8ca4-4ccf-9cc6-c3383faa493a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 00:53:33.914824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: warnings.warn(msg) Aug 26 00:53:33.914824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: Aug 26 00:53:33.916038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f392e486-8ca4-4ccf-9cc6-c3383faa493a admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:33.963130 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 777/1544] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:53:33.965171 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb990180-9c7c-4b6a-a556-db0078ecb124 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:33.965467 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb990180-9c7c-4b6a-a556-db0078ecb124 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:33.965744 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb990180-9c7c-4b6a-a556-db0078ecb124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:33.994465 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-eb990180-9c7c-4b6a-a556-db0078ecb124 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:33.995670 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-eb990180-9c7c-4b6a-a556-db0078ecb124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:34.009320 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 778/1545] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:34.011021 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c3ff1-68c4-4469-95ba-4cc3e4c936eb None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.011538 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c3ff1-68c4-4469-95ba-4cc3e4c936eb None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.012036 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c3ff1-68c4-4469-95ba-4cc3e4c936eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.014983 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 768/1546] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:53:33 2022] POST /identity/v3/users => generated 290 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:53:34.016248 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd71384-4403-4e4c-b575-63294e49d090 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.019420 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd71384-4403-4e4c-b575-63294e49d090 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.019730 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd71384-4403-4e4c-b575-63294e49d090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.038571 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-6f4c3ff1-68c4-4469-95ba-4cc3e4c936eb None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:34.040301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-6f4c3ff1-68c4-4469-95ba-4cc3e4c936eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:34.046520 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-3cd71384-4403-4e4c-b575-63294e49d090 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:34.047705 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-3cd71384-4403-4e4c-b575-63294e49d090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:34.049807 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 779/1547] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:34.050943 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7f17e-a0f9-4b61-8bfc-15de9e3d891f None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.051236 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7f17e-a0f9-4b61-8bfc-15de9e3d891f None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.051514 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7f17e-a0f9-4b61-8bfc-15de9e3d891f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.060344 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 769/1548] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:34.061600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582abf29-81b1-47d5-9c2b-874daa0b45c9 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.061883 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582abf29-81b1-47d5-9c2b-874daa0b45c9 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.062156 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582abf29-81b1-47d5-9c2b-874daa0b45c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.072971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-d1f7f17e-a0f9-4b61-8bfc-15de9e3d891f None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:34.074186 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-d1f7f17e-a0f9-4b61-8bfc-15de9e3d891f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:34.082696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 780/1549] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:34.084915 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e4032-c1e7-4c0c-8c9b-275d53d2cf73 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.085200 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e4032-c1e7-4c0c-8c9b-275d53d2cf73 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.085479 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e4032-c1e7-4c0c-8c9b-275d53d2cf73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.097213 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-582abf29-81b1-47d5-9c2b-874daa0b45c9 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:34.098563 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-582abf29-81b1-47d5-9c2b-874daa0b45c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:34.105851 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-cb1e4032-c1e7-4c0c-8c9b-275d53d2cf73 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:34.107035 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-cb1e4032-c1e7-4c0c-8c9b-275d53d2cf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:34.108093 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 770/1550] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:34.108808 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582abf29-81b1-47d5-9c2b-874daa0b45c9 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:34.118686 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582abf29-81b1-47d5-9c2b-874daa0b45c9 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:34.119143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 781/1551] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:34.120307 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b33553-5416-444f-afcc-38750eef3af4 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.120637 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b33553-5416-444f-afcc-38750eef3af4 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.120914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b33553-5416-444f-afcc-38750eef3af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.123438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17353cb0-56e9-4a53-a2b3-876d47d915a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:34.124077 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17353cb0-56e9-4a53-a2b3-876d47d915a2 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.124321 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17353cb0-56e9-4a53-a2b3-876d47d915a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.124651 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17353cb0-56e9-4a53-a2b3-876d47d915a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.125295 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17353cb0-56e9-4a53-a2b3-876d47d915a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:34.126867 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17353cb0-56e9-4a53-a2b3-876d47d915a2 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:34.137746 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 771/1552] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:53:34.138560 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17353cb0-56e9-4a53-a2b3-876d47d915a2 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:34.140892 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-b7b33553-5416-444f-afcc-38750eef3af4 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:34.142117 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-b7b33553-5416-444f-afcc-38750eef3af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:34.149879 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 782/1553] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:34.150031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17353cb0-56e9-4a53-a2b3-876d47d915a2 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:34.151056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74504871-b0fe-486a-b4da-4d3c0e8073c5 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.151381 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74504871-b0fe-486a-b4da-4d3c0e8073c5 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.151594 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74504871-b0fe-486a-b4da-4d3c0e8073c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.157977 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5206e478-954f-41d7-8e73-cfdfabe54bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:34.158572 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206e478-954f-41d7-8e73-cfdfabe54bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.158850 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206e478-954f-41d7-8e73-cfdfabe54bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.159126 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206e478-954f-41d7-8e73-cfdfabe54bc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.159760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206e478-954f-41d7-8e73-cfdfabe54bc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:53:34.160060 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206e478-954f-41d7-8e73-cfdfabe54bc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:34.161360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: WARNING py.warnings [None req-5206e478-954f-41d7-8e73-cfdfabe54bc8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 00:53:34.161360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: warnings.warn(msg) Aug 26 00:53:34.161360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: Aug 26 00:53:34.162388 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206e478-954f-41d7-8e73-cfdfabe54bc8 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:34.171764 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-74504871-b0fe-486a-b4da-4d3c0e8073c5 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:34.172992 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-74504871-b0fe-486a-b4da-4d3c0e8073c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:34.174338 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 772/1554] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:53:34.175527 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf7b96-f38b-479b-ab96-2b9052f60db6 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.175812 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf7b96-f38b-479b-ab96-2b9052f60db6 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.176086 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf7b96-f38b-479b-ab96-2b9052f60db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.182823 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 783/1555] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:34.184014 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d49df-2d35-4ded-82b4-3eb73a547d7c None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.184301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d49df-2d35-4ded-82b4-3eb73a547d7c None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.184611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d49df-2d35-4ded-82b4-3eb73a547d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.198763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-3bcf7b96-f38b-479b-ab96-2b9052f60db6 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:34.199986 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-3bcf7b96-f38b-479b-ab96-2b9052f60db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:34.203680 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-be4d49df-2d35-4ded-82b4-3eb73a547d7c None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:34.204868 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-be4d49df-2d35-4ded-82b4-3eb73a547d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:34.208064 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 773/1556] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:34.209298 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314f0e5-3362-48f6-a0d6-84bdd0077264 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.209560 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314f0e5-3362-48f6-a0d6-84bdd0077264 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.209828 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314f0e5-3362-48f6-a0d6-84bdd0077264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.215856 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 784/1557] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:34.216664 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d49df-2d35-4ded-82b4-3eb73a547d7c None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:34.226177 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d49df-2d35-4ded-82b4-3eb73a547d7c None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:34.228743 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-7314f0e5-3362-48f6-a0d6-84bdd0077264 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:34.229965 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-7314f0e5-3362-48f6-a0d6-84bdd0077264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:34.230906 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa762b0-c420-4f18-b8d5-bb6e080149ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:34.231506 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa762b0-c420-4f18-b8d5-bb6e080149ce admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.231787 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa762b0-c420-4f18-b8d5-bb6e080149ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.232059 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa762b0-c420-4f18-b8d5-bb6e080149ce admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.232730 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa762b0-c420-4f18-b8d5-bb6e080149ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:53:34.233035 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa762b0-c420-4f18-b8d5-bb6e080149ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:34.234630 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa762b0-c420-4f18-b8d5-bb6e080149ce admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:34.237883 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 774/1558] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:34.238746 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314f0e5-3362-48f6-a0d6-84bdd0077264 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:34.248440 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 785/1559] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:53:34.250194 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507ec88-22aa-496b-8af5-fcf49bc2034a None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.250330 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314f0e5-3362-48f6-a0d6-84bdd0077264 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:34.251099 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507ec88-22aa-496b-8af5-fcf49bc2034a None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.251611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507ec88-22aa-496b-8af5-fcf49bc2034a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.259441 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868a10f-5e3c-45f4-85dc-324d811383f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:34.260095 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868a10f-5e3c-45f4-85dc-324d811383f1 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.260377 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868a10f-5e3c-45f4-85dc-324d811383f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.260682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868a10f-5e3c-45f4-85dc-324d811383f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.261370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3868a10f-5e3c-45f4-85dc-324d811383f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:34.262999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3868a10f-5e3c-45f4-85dc-324d811383f1 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:34.273594 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 775/1560] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:53:34.273594 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868a10f-5e3c-45f4-85dc-324d811383f1 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:34.277981 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-e507ec88-22aa-496b-8af5-fcf49bc2034a None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:34.279226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-e507ec88-22aa-496b-8af5-fcf49bc2034a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:34.284763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868a10f-5e3c-45f4-85dc-324d811383f1 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:34.287576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 786/1561] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:34.291511 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507ec88-22aa-496b-8af5-fcf49bc2034a None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:34.291631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43be6c8-7466-4eac-92b8-4d25a254fd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:34.291631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43be6c8-7466-4eac-92b8-4d25a254fd22 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.291835 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43be6c8-7466-4eac-92b8-4d25a254fd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.291987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43be6c8-7466-4eac-92b8-4d25a254fd22 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.292645 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43be6c8-7466-4eac-92b8-4d25a254fd22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:34.294550 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43be6c8-7466-4eac-92b8-4d25a254fd22 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:34.300410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507ec88-22aa-496b-8af5-fcf49bc2034a None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:34.306002 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382ace1-5143-4dde-80f6-393667082c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:34.306602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382ace1-5143-4dde-80f6-393667082c96 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.306884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382ace1-5143-4dde-80f6-393667082c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.307154 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382ace1-5143-4dde-80f6-393667082c96 admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.307790 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382ace1-5143-4dde-80f6-393667082c96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:53:34.308087 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382ace1-5143-4dde-80f6-393667082c96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:34.309729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382ace1-5143-4dde-80f6-393667082c96 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:34.322262 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 787/1562] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:53:34.323159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382ace1-5143-4dde-80f6-393667082c96 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:34.332882 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382ace1-5143-4dde-80f6-393667082c96 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:34.337519 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9ef3b-5bc4-4f0d-af13-86c6824722dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:34.338189 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9ef3b-5bc4-4f0d-af13-86c6824722dc admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.338469 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9ef3b-5bc4-4f0d-af13-86c6824722dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.338742 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9ef3b-5bc4-4f0d-af13-86c6824722dc admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.339374 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9ef3b-5bc4-4f0d-af13-86c6824722dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:53:34.339679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9ef3b-5bc4-4f0d-af13-86c6824722dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:34.341312 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9ef3b-5bc4-4f0d-af13-86c6824722dc admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:34.353186 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 788/1563] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:53:34.354326 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824433a-9138-421b-a6f2-5d030230d4bd None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.354613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824433a-9138-421b-a6f2-5d030230d4bd None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.354887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824433a-9138-421b-a6f2-5d030230d4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.374301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-8824433a-9138-421b-a6f2-5d030230d4bd None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:34.375493 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-8824433a-9138-421b-a6f2-5d030230d4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:34.388352 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 776/1564] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:53:34 2022] POST /identity/v3/projects => generated 367 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:53:34.391311 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 789/1565] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:34.391426 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43be6c8-7466-4eac-92b8-4d25a254fd22 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:34.391545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824433a-9138-421b-a6f2-5d030230d4bd None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:34.398059 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824433a-9138-421b-a6f2-5d030230d4bd None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:34.425224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:34.442370 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:34.456686 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43be6c8-7466-4eac-92b8-4d25a254fd22 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:34.458944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175f92f-2d3f-49da-b3b8-81d63cb33941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:34.459557 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175f92f-2d3f-49da-b3b8-81d63cb33941 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.459838 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175f92f-2d3f-49da-b3b8-81d63cb33941 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.460214 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175f92f-2d3f-49da-b3b8-81d63cb33941 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.460953 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b175f92f-2d3f-49da-b3b8-81d63cb33941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:34.464071 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: WARNING py.warnings [None req-b175f92f-2d3f-49da-b3b8-81d63cb33941 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 00:53:34.464071 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: warnings.warn(msg) Aug 26 00:53:34.464071 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: Aug 26 00:53:34.464458 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b0232-01ac-4621-b40c-8aaa0795275c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:34.464589 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b175f92f-2d3f-49da-b3b8-81d63cb33941 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:34.464694 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b0232-01ac-4621-b40c-8aaa0795275c admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.464694 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b0232-01ac-4621-b40c-8aaa0795275c admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.464694 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b0232-01ac-4621-b40c-8aaa0795275c admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.465008 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b0232-01ac-4621-b40c-8aaa0795275c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:53:34.465310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b0232-01ac-4621-b40c-8aaa0795275c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:34.467245 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b0232-01ac-4621-b40c-8aaa0795275c admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:34.479891 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 777/1566] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:53:34.480783 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b0232-01ac-4621-b40c-8aaa0795275c admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:34.490743 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b0232-01ac-4621-b40c-8aaa0795275c admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:34.495385 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a8a8d-7ade-4d1e-9a8f-238a13a17d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:34.496026 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a8a8d-7ade-4d1e-9a8f-238a13a17d57 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.496307 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a8a8d-7ade-4d1e-9a8f-238a13a17d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.496615 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a8a8d-7ade-4d1e-9a8f-238a13a17d57 admin admin] PATH_INFO: `/v3/projects/0527db336d294c969ec669c8aa56ac2f/users/b390ac3085984efc909f44c7152576ff/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.511656 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a8a8d-7ade-4d1e-9a8f-238a13a17d57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0527db336d294c969ec669c8aa56ac2f, user_id=b390ac3085984efc909f44c7152576ff, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:34.514334 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a8a8d-7ade-4d1e-9a8f-238a13a17d57 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:34.542237 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 790/1567] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:53:34 2022] POST /identity/v3/projects => generated 375 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:53:34.542862 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175f92f-2d3f-49da-b3b8-81d63cb33941 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:34.603117 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 778/1568] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:53:34 2022] PUT /identity/v3/projects/0527db336d294c969ec669c8aa56ac2f/users/b390ac3085984efc909f44c7152576ff/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:53:34.603810 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175f92f-2d3f-49da-b3b8-81d63cb33941 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:34.603918 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a8a8d-7ade-4d1e-9a8f-238a13a17d57 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:34.669982 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8d9a3-5845-4253-8da9-1fa66ea2e2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:34.670607 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8d9a3-5845-4253-8da9-1fa66ea2e2bd admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.670885 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8d9a3-5845-4253-8da9-1fa66ea2e2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.670991 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a8a8d-7ade-4d1e-9a8f-238a13a17d57 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:34.671157 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8d9a3-5845-4253-8da9-1fa66ea2e2bd admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.671803 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8d9a3-5845-4253-8da9-1fa66ea2e2bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:53:34.672105 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8d9a3-5845-4253-8da9-1fa66ea2e2bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:34.673750 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8d9a3-5845-4253-8da9-1fa66ea2e2bd admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:34.675872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704aa182-905d-44ae-8d32-b902acfaaaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:34.676495 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704aa182-905d-44ae-8d32-b902acfaaaea admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.676839 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704aa182-905d-44ae-8d32-b902acfaaaea admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.677113 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704aa182-905d-44ae-8d32-b902acfaaaea admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.678329 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704aa182-905d-44ae-8d32-b902acfaaaea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:34.680233 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704aa182-905d-44ae-8d32-b902acfaaaea admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:34.687474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 791/1569] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:53:34.687474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8d9a3-5845-4253-8da9-1fa66ea2e2bd admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:34.703700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8d9a3-5845-4253-8da9-1fa66ea2e2bd admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:34.709099 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f875d-9cec-4d65-8319-d2db77321d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:34.711052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f875d-9cec-4d65-8319-d2db77321d9d admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.711052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f875d-9cec-4d65-8319-d2db77321d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.711052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f875d-9cec-4d65-8319-d2db77321d9d admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.711052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1f875d-9cec-4d65-8319-d2db77321d9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:34.713126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1f875d-9cec-4d65-8319-d2db77321d9d admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:34.798416 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 779/1570] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:53:34 2022] POST /identity/v3/projects => generated 391 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:53:34.799262 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704aa182-905d-44ae-8d32-b902acfaaaea admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:34.859510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704aa182-905d-44ae-8d32-b902acfaaaea admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:34.864382 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201c851-e892-45f4-8ffc-fe95bdff10b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:34.865099 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201c851-e892-45f4-8ffc-fe95bdff10b4 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.866030 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201c851-e892-45f4-8ffc-fe95bdff10b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.866309 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201c851-e892-45f4-8ffc-fe95bdff10b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.866964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201c851-e892-45f4-8ffc-fe95bdff10b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:53:34.867279 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201c851-e892-45f4-8ffc-fe95bdff10b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:34.868937 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201c851-e892-45f4-8ffc-fe95bdff10b4 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:34.873066 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 792/1571] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:53:34 2022] POST /identity/v3/projects => generated 387 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:53:34.873878 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f875d-9cec-4d65-8319-d2db77321d9d admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:34.881790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 780/1572] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:53:34.883229 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201c851-e892-45f4-8ffc-fe95bdff10b4 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:34.892930 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201c851-e892-45f4-8ffc-fe95bdff10b4 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:34.906229 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:34.926913 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:34.941397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f875d-9cec-4d65-8319-d2db77321d9d admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:34.946124 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8052e481-fb6e-4972-9ffe-e533b6cf8f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:34.946736 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052e481-fb6e-4972-9ffe-e533b6cf8f9b admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.947013 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052e481-fb6e-4972-9ffe-e533b6cf8f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.947284 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052e481-fb6e-4972-9ffe-e533b6cf8f9b admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.947904 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8052e481-fb6e-4972-9ffe-e533b6cf8f9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:34.949906 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8052e481-fb6e-4972-9ffe-e533b6cf8f9b admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:34.952367 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae66a88-44bd-4f32-a6ff-dbf6660c1608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:34.953005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae66a88-44bd-4f32-a6ff-dbf6660c1608 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:34.953289 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae66a88-44bd-4f32-a6ff-dbf6660c1608 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:34.953566 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae66a88-44bd-4f32-a6ff-dbf6660c1608 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:34.954192 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae66a88-44bd-4f32-a6ff-dbf6660c1608 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:34.956091 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae66a88-44bd-4f32-a6ff-dbf6660c1608 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:35.037329 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 793/1573] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:53:34 2022] POST /identity/v3/users => generated 284 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:53:35.037951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8052e481-fb6e-4972-9ffe-e533b6cf8f9b admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:35.038161 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 781/1574] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:53:34 2022] POST /identity/v3/projects => generated 377 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:53:35.039403 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31675cf5-d4fa-4faf-a9a0-918e1f66c01a None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:35.039681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31675cf5-d4fa-4faf-a9a0-918e1f66c01a None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:35.043510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31675cf5-d4fa-4faf-a9a0-918e1f66c01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:35.047947 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8052e481-fb6e-4972-9ffe-e533b6cf8f9b admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:35.068061 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-31675cf5-d4fa-4faf-a9a0-918e1f66c01a None None] MFA Rules not processed for user `b390ac3085984efc909f44c7152576ff`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:35.069918 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-31675cf5-d4fa-4faf-a9a0-918e1f66c01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:35.105342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760dc0a-ec7b-4579-8b09-67f9aef8bd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:35.105957 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760dc0a-ec7b-4579-8b09-67f9aef8bd0b admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:35.106231 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760dc0a-ec7b-4579-8b09-67f9aef8bd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:35.106498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760dc0a-ec7b-4579-8b09-67f9aef8bd0b admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:35.107119 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8760dc0a-ec7b-4579-8b09-67f9aef8bd0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:35.109077 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8760dc0a-ec7b-4579-8b09-67f9aef8bd0b admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:35.132623 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 782/1575] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:35.133443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31675cf5-d4fa-4faf-a9a0-918e1f66c01a None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:35.143243 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31675cf5-d4fa-4faf-a9a0-918e1f66c01a None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:35.147728 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e364285-7e44-4f16-9803-323591dab4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:35.148309 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e364285-7e44-4f16-9803-323591dab4b0 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:35.148618 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e364285-7e44-4f16-9803-323591dab4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:35.148894 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e364285-7e44-4f16-9803-323591dab4b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:35.149507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e364285-7e44-4f16-9803-323591dab4b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:35.151406 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e364285-7e44-4f16-9803-323591dab4b0 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:35.202956 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 794/1576] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:53:35 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:53:35.203790 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760dc0a-ec7b-4579-8b09-67f9aef8bd0b admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:35.213277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760dc0a-ec7b-4579-8b09-67f9aef8bd0b admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:35.217736 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a728da-f698-421b-949a-cc1f029ffcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:35.218412 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a728da-f698-421b-949a-cc1f029ffcdb admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:35.218693 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a728da-f698-421b-949a-cc1f029ffcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:35.218961 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a728da-f698-421b-949a-cc1f029ffcdb admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:35.219564 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a728da-f698-421b-949a-cc1f029ffcdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:35.221496 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a728da-f698-421b-949a-cc1f029ffcdb admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:35.226788 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 783/1577] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:53:35 2022] POST /identity/v3/projects => generated 373 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:53:35.228734 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e364285-7e44-4f16-9803-323591dab4b0 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:35.292265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 795/1578] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:53:35 2022] POST /identity/v3/users => generated 296 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:53:35.292265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a728da-f698-421b-949a-cc1f029ffcdb admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:35.292618 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e364285-7e44-4f16-9803-323591dab4b0 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:35.297499 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9d0ff-6f5a-4110-8053-89be45757d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:35.298174 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9d0ff-6f5a-4110-8053-89be45757d0a admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:35.298456 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9d0ff-6f5a-4110-8053-89be45757d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:35.298727 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9d0ff-6f5a-4110-8053-89be45757d0a admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:35.299352 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9d0ff-6f5a-4110-8053-89be45757d0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:35.299465 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a728da-f698-421b-949a-cc1f029ffcdb admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:35.301364 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9d0ff-6f5a-4110-8053-89be45757d0a admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:35.304617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0035f41-98e1-4aba-acfa-9990dc92d935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:35.305211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0035f41-98e1-4aba-acfa-9990dc92d935 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:35.305494 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0035f41-98e1-4aba-acfa-9990dc92d935 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:35.305768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0035f41-98e1-4aba-acfa-9990dc92d935 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:35.306414 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0035f41-98e1-4aba-acfa-9990dc92d935 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:35.308296 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0035f41-98e1-4aba-acfa-9990dc92d935 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:35.374793 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:35.380214 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 784/1579] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:53:35 2022] POST /identity/v3/users => generated 294 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:53:35.385010 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9d0ff-6f5a-4110-8053-89be45757d0a admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:35.391224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9d0ff-6f5a-4110-8053-89be45757d0a admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:35.395784 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2dfa55-8af7-4953-9ccd-63e8f908378c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:35.396930 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2dfa55-8af7-4953-9ccd-63e8f908378c admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:35.397225 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2dfa55-8af7-4953-9ccd-63e8f908378c admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:35.397499 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2dfa55-8af7-4953-9ccd-63e8f908378c admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:35.398129 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2dfa55-8af7-4953-9ccd-63e8f908378c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:35.399696 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2dfa55-8af7-4953-9ccd-63e8f908378c admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:35.409585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 785/1580] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:53:35.410653 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2dfa55-8af7-4953-9ccd-63e8f908378c admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:35.416798 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 796/1581] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:53:35 2022] POST /identity/v3/projects => generated 371 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:53:35.417867 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0035f41-98e1-4aba-acfa-9990dc92d935 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:35.420245 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2dfa55-8af7-4953-9ccd-63e8f908378c admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:35.477625 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce3011-6867-4d6d-9dae-ca8457132044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:35.478241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce3011-6867-4d6d-9dae-ca8457132044 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:35.478525 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce3011-6867-4d6d-9dae-ca8457132044 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:35.478801 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce3011-6867-4d6d-9dae-ca8457132044 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:35.479425 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ce3011-6867-4d6d-9dae-ca8457132044 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:35.481461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ce3011-6867-4d6d-9dae-ca8457132044 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:35.486765 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0035f41-98e1-4aba-acfa-9990dc92d935 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:35.491366 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3bd85d-1c14-45a7-8118-caf2eb9cddab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:35.491979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bd85d-1c14-45a7-8118-caf2eb9cddab admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:35.492254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bd85d-1c14-45a7-8118-caf2eb9cddab admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:35.492555 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bd85d-1c14-45a7-8118-caf2eb9cddab admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:35.493183 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3bd85d-1c14-45a7-8118-caf2eb9cddab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:35.495091 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3bd85d-1c14-45a7-8118-caf2eb9cddab admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:35.533340 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 786/1582] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:53:35 2022] POST /identity/v3/users => generated 289 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:53:35.534176 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce3011-6867-4d6d-9dae-ca8457132044 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:35.543763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce3011-6867-4d6d-9dae-ca8457132044 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:35.548241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdf04b-ac40-46db-8a6e-93011b5a0eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:35.548859 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdf04b-ac40-46db-8a6e-93011b5a0eee admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:35.551543 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdf04b-ac40-46db-8a6e-93011b5a0eee admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:35.551543 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdf04b-ac40-46db-8a6e-93011b5a0eee admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:35.551543 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbdf04b-ac40-46db-8a6e-93011b5a0eee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:35.551543 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbdf04b-ac40-46db-8a6e-93011b5a0eee admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:35.551991 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 797/1583] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:53:35 2022] POST /identity/v3/projects => generated 381 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:53:35.552850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3bd85d-1c14-45a7-8118-caf2eb9cddab admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:35.562179 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 787/1584] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:53:35.562998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdf04b-ac40-46db-8a6e-93011b5a0eee admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:35.565678 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:35.572872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdf04b-ac40-46db-8a6e-93011b5a0eee admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:35.617356 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3bd85d-1c14-45a7-8118-caf2eb9cddab admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:35.622038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d2cc5-0dc8-49bd-b6b6-2e89350b843f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:35.622683 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d2cc5-0dc8-49bd-b6b6-2e89350b843f admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:35.622965 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d2cc5-0dc8-49bd-b6b6-2e89350b843f admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:35.623245 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d2cc5-0dc8-49bd-b6b6-2e89350b843f admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:35.623875 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d2cc5-0dc8-49bd-b6b6-2e89350b843f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:35.625915 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d2cc5-0dc8-49bd-b6b6-2e89350b843f admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:35.629805 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f119be5-d552-4ebd-8b7f-36f4e7cdfdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:35.631552 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f119be5-d552-4ebd-8b7f-36f4e7cdfdcd admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:35.632259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f119be5-d552-4ebd-8b7f-36f4e7cdfdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:35.632794 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f119be5-d552-4ebd-8b7f-36f4e7cdfdcd admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:35.633632 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f119be5-d552-4ebd-8b7f-36f4e7cdfdcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:35.635974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f119be5-d552-4ebd-8b7f-36f4e7cdfdcd admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:35.647635 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 788/1585] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:53:35.648458 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f119be5-d552-4ebd-8b7f-36f4e7cdfdcd admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:35.658067 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f119be5-d552-4ebd-8b7f-36f4e7cdfdcd admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:35.662892 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef64497-5ab2-41d4-add7-90897b4eb433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:35.663465 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef64497-5ab2-41d4-add7-90897b4eb433 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:35.663744 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef64497-5ab2-41d4-add7-90897b4eb433 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:35.664020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef64497-5ab2-41d4-add7-90897b4eb433 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:35.664666 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef64497-5ab2-41d4-add7-90897b4eb433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:35.666227 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef64497-5ab2-41d4-add7-90897b4eb433 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:35.675762 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 789/1586] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:53:35.676558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef64497-5ab2-41d4-add7-90897b4eb433 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:35.686533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef64497-5ab2-41d4-add7-90897b4eb433 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:35.691224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a63ee-f42b-4554-beea-4e7a2c7c5b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:35.691828 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a63ee-f42b-4554-beea-4e7a2c7c5b15 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:35.692110 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a63ee-f42b-4554-beea-4e7a2c7c5b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:35.692415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a63ee-f42b-4554-beea-4e7a2c7c5b15 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:35.693780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69a63ee-f42b-4554-beea-4e7a2c7c5b15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:35.695415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69a63ee-f42b-4554-beea-4e7a2c7c5b15 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:35.705065 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 798/1587] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:53:35 2022] POST /identity/v3/users => generated 287 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:53:35.710421 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d2cc5-0dc8-49bd-b6b6-2e89350b843f admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:35.710531 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 790/1588] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:53:35.710531 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a63ee-f42b-4554-beea-4e7a2c7c5b15 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:35.718676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d2cc5-0dc8-49bd-b6b6-2e89350b843f admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:35.718858 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a63ee-f42b-4554-beea-4e7a2c7c5b15 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:35.723229 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef91fc0-678b-4bd9-9459-e2d826da4562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:35.723769 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84839a27-643c-45c3-9c61-f4ad66f78402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:35.723910 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef91fc0-678b-4bd9-9459-e2d826da4562 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:35.724098 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef91fc0-678b-4bd9-9459-e2d826da4562 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:35.724397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef91fc0-678b-4bd9-9459-e2d826da4562 admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:35.724561 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84839a27-643c-45c3-9c61-f4ad66f78402 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:35.724743 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84839a27-643c-45c3-9c61-f4ad66f78402 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:35.724982 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84839a27-643c-45c3-9c61-f4ad66f78402 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:35.725158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef91fc0-678b-4bd9-9459-e2d826da4562 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:35.725613 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84839a27-643c-45c3-9c61-f4ad66f78402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:35.727035 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef91fc0-678b-4bd9-9459-e2d826da4562 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:35.727482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84839a27-643c-45c3-9c61-f4ad66f78402 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:35.737687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 791/1589] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:53:35.738482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84839a27-643c-45c3-9c61-f4ad66f78402 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:35.748530 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84839a27-643c-45c3-9c61-f4ad66f78402 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:35.753076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd7205-61a8-4b7b-89c6-e065b492e2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:35.753667 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd7205-61a8-4b7b-89c6-e065b492e2ec admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:35.753975 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd7205-61a8-4b7b-89c6-e065b492e2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:35.754278 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd7205-61a8-4b7b-89c6-e065b492e2ec admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:35.754955 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd7205-61a8-4b7b-89c6-e065b492e2ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:35.757169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd7205-61a8-4b7b-89c6-e065b492e2ec admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:35.844230 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 799/1590] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:53:35 2022] POST /identity/v3/users => generated 286 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:53:35.845308 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef91fc0-678b-4bd9-9459-e2d826da4562 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:35.855818 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef91fc0-678b-4bd9-9459-e2d826da4562 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:35.860400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce46be-86e3-4bbf-88d9-5bcb6a8902f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:35.861020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce46be-86e3-4bbf-88d9-5bcb6a8902f5 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:35.861303 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce46be-86e3-4bbf-88d9-5bcb6a8902f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:35.861578 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce46be-86e3-4bbf-88d9-5bcb6a8902f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:35.862229 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce46be-86e3-4bbf-88d9-5bcb6a8902f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:35.863856 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce46be-86e3-4bbf-88d9-5bcb6a8902f5 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:35.874035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 792/1591] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:53:35 2022] POST /identity/v3/users => generated 297 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:53:35.874035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd7205-61a8-4b7b-89c6-e065b492e2ec admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:35.879771 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 800/1592] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:53:35.879771 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce46be-86e3-4bbf-88d9-5bcb6a8902f5 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:35.883456 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd7205-61a8-4b7b-89c6-e065b492e2ec admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:35.887660 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce46be-86e3-4bbf-88d9-5bcb6a8902f5 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:35.888104 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f33348-283c-4309-ba31-4f5e99e835c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:35.888731 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f33348-283c-4309-ba31-4f5e99e835c8 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:35.889007 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f33348-283c-4309-ba31-4f5e99e835c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:35.889389 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f33348-283c-4309-ba31-4f5e99e835c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:35.890069 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f33348-283c-4309-ba31-4f5e99e835c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:35.891724 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f33348-283c-4309-ba31-4f5e99e835c8 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:35.892438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718d118-abb9-4e5a-97dd-9abaa7f372b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:35.897816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718d118-abb9-4e5a-97dd-9abaa7f372b5 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:35.897816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718d118-abb9-4e5a-97dd-9abaa7f372b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:35.897816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718d118-abb9-4e5a-97dd-9abaa7f372b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:35.897816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6718d118-abb9-4e5a-97dd-9abaa7f372b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:35.897816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6718d118-abb9-4e5a-97dd-9abaa7f372b5 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:35.902622 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 793/1593] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:53:35.903427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f33348-283c-4309-ba31-4f5e99e835c8 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:35.907544 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 801/1594] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:53:35.908359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718d118-abb9-4e5a-97dd-9abaa7f372b5 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:35.909046 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:35.916132 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f33348-283c-4309-ba31-4f5e99e835c8 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:35.919030 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718d118-abb9-4e5a-97dd-9abaa7f372b5 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:35.920877 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec414fa-72b4-49e9-bd75-7bf4047184cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:35.921466 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec414fa-72b4-49e9-bd75-7bf4047184cc admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:35.921744 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec414fa-72b4-49e9-bd75-7bf4047184cc admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:35.922016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec414fa-72b4-49e9-bd75-7bf4047184cc admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:35.922646 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec414fa-72b4-49e9-bd75-7bf4047184cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:35.924209 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec414fa-72b4-49e9-bd75-7bf4047184cc admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:35.924344 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e3087-d2bc-4570-b52c-d5fda2742d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:35.924962 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e3087-d2bc-4570-b52c-d5fda2742d59 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:35.925300 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e3087-d2bc-4570-b52c-d5fda2742d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:35.925565 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e3087-d2bc-4570-b52c-d5fda2742d59 admin admin] PATH_INFO: `/v3/projects/b0540b60f8e04a0584625e34fbcafa50/users/877072c4f9d24f5d9cdcad35817101d8/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:35.934241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 794/1595] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:53:35.935048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec414fa-72b4-49e9-bd75-7bf4047184cc admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:35.945011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec414fa-72b4-49e9-bd75-7bf4047184cc admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:35.945514 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5e3087-d2bc-4570-b52c-d5fda2742d59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0540b60f8e04a0584625e34fbcafa50, user_id=877072c4f9d24f5d9cdcad35817101d8, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:35.946701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: WARNING py.warnings [None req-5c5e3087-d2bc-4570-b52c-d5fda2742d59 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 00:53:35.946701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: warnings.warn(msg) Aug 26 00:53:35.946701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: Aug 26 00:53:35.948822 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5e3087-d2bc-4570-b52c-d5fda2742d59 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:35.950147 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5740324-2bd0-4b4a-a66c-d633401c0782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:35.951048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5740324-2bd0-4b4a-a66c-d633401c0782 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:35.951344 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5740324-2bd0-4b4a-a66c-d633401c0782 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:35.952073 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5740324-2bd0-4b4a-a66c-d633401c0782 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:35.952821 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5740324-2bd0-4b4a-a66c-d633401c0782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:35.954625 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5740324-2bd0-4b4a-a66c-d633401c0782 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:35.965206 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 795/1596] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:53:35.966147 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5740324-2bd0-4b4a-a66c-d633401c0782 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:35.975649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5740324-2bd0-4b4a-a66c-d633401c0782 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:35.980653 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33ef1f-777f-40dc-9607-f673eece1137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:35.981222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33ef1f-777f-40dc-9607-f673eece1137 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:35.981508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33ef1f-777f-40dc-9607-f673eece1137 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:35.981815 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33ef1f-777f-40dc-9607-f673eece1137 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:35.982432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33ef1f-777f-40dc-9607-f673eece1137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:35.984007 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33ef1f-777f-40dc-9607-f673eece1137 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:35.994616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 796/1597] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:53:35.995627 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33ef1f-777f-40dc-9607-f673eece1137 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:36.004878 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33ef1f-777f-40dc-9607-f673eece1137 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:36.010599 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1a177-160c-4092-bf7e-e34e72ebaebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:36.011159 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1a177-160c-4092-bf7e-e34e72ebaebd admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:36.011442 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1a177-160c-4092-bf7e-e34e72ebaebd admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:36.011715 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1a177-160c-4092-bf7e-e34e72ebaebd admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:36.012331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1a177-160c-4092-bf7e-e34e72ebaebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:36.013922 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1a177-160c-4092-bf7e-e34e72ebaebd admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:36.023562 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 797/1598] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:53:36.024457 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1a177-160c-4092-bf7e-e34e72ebaebd admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:36.030261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 802/1599] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:53:35 2022] PUT /identity/v3/projects/b0540b60f8e04a0584625e34fbcafa50/users/877072c4f9d24f5d9cdcad35817101d8/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:53:36.031188 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e3087-d2bc-4570-b52c-d5fda2742d59 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:36.040780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1a177-160c-4092-bf7e-e34e72ebaebd admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:36.096944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef5134-c5e3-416b-99be-f67a1c5a4318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:36.097464 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e3087-d2bc-4570-b52c-d5fda2742d59 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:36.097612 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef5134-c5e3-416b-99be-f67a1c5a4318 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:36.097895 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef5134-c5e3-416b-99be-f67a1c5a4318 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:36.098171 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef5134-c5e3-416b-99be-f67a1c5a4318 admin admin] PATH_INFO: `/v3/projects/5219060f4c184521919b017e8435eca5/users/0175a2b440704d51aa14607a2e20e36a/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:36.102982 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38549c8a-4b27-414a-84cd-b0023f2b6c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:36.103798 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38549c8a-4b27-414a-84cd-b0023f2b6c0f admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:36.104102 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38549c8a-4b27-414a-84cd-b0023f2b6c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:36.104373 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38549c8a-4b27-414a-84cd-b0023f2b6c0f admin admin] PATH_INFO: `/v3/projects/368f6a9e030e4aff9ef6559a6a165975/users/0cc36def73aa4bedb9e79d4cf11f5e8c/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:36.113000 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef5134-c5e3-416b-99be-f67a1c5a4318 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5219060f4c184521919b017e8435eca5, user_id=0175a2b440704d51aa14607a2e20e36a, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:36.115887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef5134-c5e3-416b-99be-f67a1c5a4318 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:36.119004 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38549c8a-4b27-414a-84cd-b0023f2b6c0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=368f6a9e030e4aff9ef6559a6a165975, user_id=0cc36def73aa4bedb9e79d4cf11f5e8c, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:36.121933 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38549c8a-4b27-414a-84cd-b0023f2b6c0f admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:36.219222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 798/1600] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:53:36 2022] PUT /identity/v3/projects/5219060f4c184521919b017e8435eca5/users/0175a2b440704d51aa14607a2e20e36a/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:53:36.220105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef5134-c5e3-416b-99be-f67a1c5a4318 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:36.254301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 803/1601] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:53:36 2022] PUT /identity/v3/projects/368f6a9e030e4aff9ef6559a6a165975/users/0cc36def73aa4bedb9e79d4cf11f5e8c/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:53:36.255247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38549c8a-4b27-414a-84cd-b0023f2b6c0f admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:36.277208 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:36.282123 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef5134-c5e3-416b-99be-f67a1c5a4318 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:36.317423 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38549c8a-4b27-414a-84cd-b0023f2b6c0f admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:36.323142 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac72b4-d3dd-45e5-a620-98dbd40ae57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:36.323767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac72b4-d3dd-45e5-a620-98dbd40ae57d admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:36.324047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac72b4-d3dd-45e5-a620-98dbd40ae57d admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:36.324323 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac72b4-d3dd-45e5-a620-98dbd40ae57d admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:36.325000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac72b4-d3dd-45e5-a620-98dbd40ae57d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:36.326609 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac72b4-d3dd-45e5-a620-98dbd40ae57d admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:36.326963 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:36.338134 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 804/1602] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:53:36.340595 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac72b4-d3dd-45e5-a620-98dbd40ae57d admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:36.340722 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca212181-daec-47d7-a535-3efbd5977203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:36.340722 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca212181-daec-47d7-a535-3efbd5977203 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:36.340722 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca212181-daec-47d7-a535-3efbd5977203 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:36.341140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca212181-daec-47d7-a535-3efbd5977203 admin admin] PATH_INFO: `/v3/projects/ec39c32378ac4017ad50aa6f930e514b/users/34240c306dc54f0c8758dbb4b4525376/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:36.349478 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac72b4-d3dd-45e5-a620-98dbd40ae57d admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:36.354066 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5459c7-86b0-4982-8afa-d4ed112efb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:36.354703 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5459c7-86b0-4982-8afa-d4ed112efb81 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:36.355275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5459c7-86b0-4982-8afa-d4ed112efb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:36.355543 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5459c7-86b0-4982-8afa-d4ed112efb81 admin admin] PATH_INFO: `/v3/projects/032ed1b5aea842edb0bcae9d23810786/users/1a425027e59242bba6b05407d19cac82/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:36.356976 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca212181-daec-47d7-a535-3efbd5977203 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec39c32378ac4017ad50aa6f930e514b, user_id=34240c306dc54f0c8758dbb4b4525376, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:36.359759 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca212181-daec-47d7-a535-3efbd5977203 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:36.370640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5459c7-86b0-4982-8afa-d4ed112efb81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=032ed1b5aea842edb0bcae9d23810786, user_id=1a425027e59242bba6b05407d19cac82, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:36.375165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5459c7-86b0-4982-8afa-d4ed112efb81 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:36.465007 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 799/1604] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:53:36 2022] PUT /identity/v3/projects/ec39c32378ac4017ad50aa6f930e514b/users/34240c306dc54f0c8758dbb4b4525376/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:53:36.465173 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 805/1604] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:53:36 2022] PUT /identity/v3/projects/032ed1b5aea842edb0bcae9d23810786/users/1a425027e59242bba6b05407d19cac82/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:53:36.465653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5459c7-86b0-4982-8afa-d4ed112efb81 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:36.465797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca212181-daec-47d7-a535-3efbd5977203 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:36.527475 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca212181-daec-47d7-a535-3efbd5977203 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:36.532061 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f4de5-b86d-404f-a6c5-992cf86255fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:36.532702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f4de5-b86d-404f-a6c5-992cf86255fa admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:36.532987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f4de5-b86d-404f-a6c5-992cf86255fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:36.533098 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5459c7-86b0-4982-8afa-d4ed112efb81 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:36.533260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f4de5-b86d-404f-a6c5-992cf86255fa admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:36.533898 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f4de5-b86d-404f-a6c5-992cf86255fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:36.535873 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f4de5-b86d-404f-a6c5-992cf86255fa admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:36.537965 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87341789-349d-42eb-bf02-150700428d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:36.538623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87341789-349d-42eb-bf02-150700428d08 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:36.538906 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87341789-349d-42eb-bf02-150700428d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:36.539198 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87341789-349d-42eb-bf02-150700428d08 admin admin] PATH_INFO: `/v3/projects/b32342e98f1e4d6d8279e7a7d019c0fa/users/6e1aff1e456a4396a7c8a76141ed6324/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:36.545775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 800/1605] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:53:36.545775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29d42a-2ff2-4cda-b93b-04f8d0678a7d None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:36.545775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29d42a-2ff2-4cda-b93b-04f8d0678a7d None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:36.545775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29d42a-2ff2-4cda-b93b-04f8d0678a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:36.555035 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87341789-349d-42eb-bf02-150700428d08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b32342e98f1e4d6d8279e7a7d019c0fa, user_id=6e1aff1e456a4396a7c8a76141ed6324, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:36.558660 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87341789-349d-42eb-bf02-150700428d08 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:36.575342 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-9f29d42a-2ff2-4cda-b93b-04f8d0678a7d None None] MFA Rules not processed for user `877072c4f9d24f5d9cdcad35817101d8`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:36.576581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-9f29d42a-2ff2-4cda-b93b-04f8d0678a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:36.630361 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 806/1606] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:53:36 2022] PUT /identity/v3/projects/b32342e98f1e4d6d8279e7a7d019c0fa/users/6e1aff1e456a4396a7c8a76141ed6324/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:53:36.632521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e449d-025a-4474-b37b-9bd9556de032 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:36.632861 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e449d-025a-4474-b37b-9bd9556de032 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:36.633315 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e449d-025a-4474-b37b-9bd9556de032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:36.639907 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 801/1607] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:36.641319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e0829-cb3b-40c8-8694-9d4a654c15d1 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:36.641615 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e0829-cb3b-40c8-8694-9d4a654c15d1 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:36.641895 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e0829-cb3b-40c8-8694-9d4a654c15d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:36.659369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-1e5e449d-025a-4474-b37b-9bd9556de032 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:36.660698 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-375e0829-cb3b-40c8-8694-9d4a654c15d1 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:36.661005 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-1e5e449d-025a-4474-b37b-9bd9556de032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:36.661902 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-375e0829-cb3b-40c8-8694-9d4a654c15d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:36.713659 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:36.722895 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 802/1608] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:36.723774 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e0829-cb3b-40c8-8694-9d4a654c15d1 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:36.729784 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 807/1609] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:36.731263 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73982092-b2ea-4524-9502-e2597f8cb95c None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:36.731707 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73982092-b2ea-4524-9502-e2597f8cb95c None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:36.732122 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73982092-b2ea-4524-9502-e2597f8cb95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:36.734034 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e0829-cb3b-40c8-8694-9d4a654c15d1 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:36.739052 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2749711-9925-447b-82b3-503f95104e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:36.739709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2749711-9925-447b-82b3-503f95104e15 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:36.740597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2749711-9925-447b-82b3-503f95104e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:36.740809 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2749711-9925-447b-82b3-503f95104e15 admin admin] PATH_INFO: `/v3/projects/bd196a920ed04c0bac853f0556b4233d/users/5e69fd17e416401e84aa51537eed540f/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:36.755643 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2749711-9925-447b-82b3-503f95104e15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd196a920ed04c0bac853f0556b4233d, user_id=5e69fd17e416401e84aa51537eed540f, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:36.756446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-73982092-b2ea-4524-9502-e2597f8cb95c None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:36.757684 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-73982092-b2ea-4524-9502-e2597f8cb95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:36.758816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2749711-9925-447b-82b3-503f95104e15 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:36.765285 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 808/1610] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:36.766456 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0a221-78cf-4b7c-bb93-220879f38e57 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:36.766733 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0a221-78cf-4b7c-bb93-220879f38e57 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:36.767009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0a221-78cf-4b7c-bb93-220879f38e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:36.785540 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-1bb0a221-78cf-4b7c-bb93-220879f38e57 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:36.786652 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-1bb0a221-78cf-4b7c-bb93-220879f38e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:36.795818 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 809/1611] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:36.796598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0a221-78cf-4b7c-bb93-220879f38e57 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:36.801069 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:36.806988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0a221-78cf-4b7c-bb93-220879f38e57 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:36.811619 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaad093-667d-4461-afed-28d547a19b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:36.816396 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaad093-667d-4461-afed-28d547a19b9d admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:36.816396 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaad093-667d-4461-afed-28d547a19b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:36.816396 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaad093-667d-4461-afed-28d547a19b9d admin admin] PATH_INFO: `/v3/projects/ce7591f88899478b9bc71cd16ef37897/users/e68a4a654a1e4a49b9e59cff12b68e72/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:36.832998 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaad093-667d-4461-afed-28d547a19b9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce7591f88899478b9bc71cd16ef37897, user_id=e68a4a654a1e4a49b9e59cff12b68e72, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:36.837490 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaad093-667d-4461-afed-28d547a19b9d admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:36.842713 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 803/1612] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:53:36 2022] PUT /identity/v3/projects/bd196a920ed04c0bac853f0556b4233d/users/5e69fd17e416401e84aa51537eed540f/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:53:36.843839 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2749711-9925-447b-82b3-503f95104e15 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:36.906386 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2749711-9925-447b-82b3-503f95104e15 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:36.911412 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3b21d-fca7-4b9d-a0cd-fe84491f33af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:36.912567 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3b21d-fca7-4b9d-a0cd-fe84491f33af admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:36.912876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3b21d-fca7-4b9d-a0cd-fe84491f33af admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:36.913151 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3b21d-fca7-4b9d-a0cd-fe84491f33af admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:36.913793 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc3b21d-fca7-4b9d-a0cd-fe84491f33af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:36.915423 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc3b21d-fca7-4b9d-a0cd-fe84491f33af admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:36.923156 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 810/1613] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:53:36 2022] PUT /identity/v3/projects/ce7591f88899478b9bc71cd16ef37897/users/e68a4a654a1e4a49b9e59cff12b68e72/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:53:36.929289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaad093-667d-4461-afed-28d547a19b9d admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:36.929430 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 804/1614] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:53:36.929430 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3b21d-fca7-4b9d-a0cd-fe84491f33af admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:36.982428 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaad093-667d-4461-afed-28d547a19b9d admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:36.986076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3b21d-fca7-4b9d-a0cd-fe84491f33af admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:36.986212 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c5367-05eb-443a-adbc-c631ba556dfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:36.986714 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c5367-05eb-443a-adbc-c631ba556dfc service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:36.987058 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c5367-05eb-443a-adbc-c631ba556dfc service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:36.987338 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c5367-05eb-443a-adbc-c631ba556dfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:37.034885 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f14c21-a4bd-49ce-ba19-aad5b48a9302 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:37.036110 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f14c21-a4bd-49ce-ba19-aad5b48a9302 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:37.036425 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f14c21-a4bd-49ce-ba19-aad5b48a9302 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:37.036602 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f14c21-a4bd-49ce-ba19-aad5b48a9302 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:37.044308 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c5367-05eb-443a-adbc-c631ba556dfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:37.045924 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c5367-05eb-443a-adbc-c631ba556dfc service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:37.055138 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 811/1615] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:53:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:53:37.055964 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c5367-05eb-443a-adbc-c631ba556dfc service cinder] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:37.057344 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-416c5367-05eb-443a-adbc-c631ba556dfc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:37.069451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c5367-05eb-443a-adbc-c631ba556dfc service cinder] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:37.074351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c60a24-df8a-4cb0-9b7a-1b346bdc767b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:37.074958 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c60a24-df8a-4cb0-9b7a-1b346bdc767b service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:37.075237 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c60a24-df8a-4cb0-9b7a-1b346bdc767b service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:37.075507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c60a24-df8a-4cb0-9b7a-1b346bdc767b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:37.080109 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:37.101072 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f14c21-a4bd-49ce-ba19-aad5b48a9302 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:37.102559 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f14c21-a4bd-49ce-ba19-aad5b48a9302 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:37.112081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 805/1616] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:53:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:53:37.112989 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f14c21-a4bd-49ce-ba19-aad5b48a9302 service neutron] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:37.122453 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f14c21-a4bd-49ce-ba19-aad5b48a9302 service neutron] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:37.127535 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aee566-ce5e-490a-8228-115788260f4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:37.129083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aee566-ce5e-490a-8228-115788260f4e service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:37.129934 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aee566-ce5e-490a-8228-115788260f4e service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:37.130280 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aee566-ce5e-490a-8228-115788260f4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:37.143240 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aee566-ce5e-490a-8228-115788260f4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:37.143395 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c60a24-df8a-4cb0-9b7a-1b346bdc767b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:37.143510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aee566-ce5e-490a-8228-115788260f4e service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:37.144370 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c60a24-df8a-4cb0-9b7a-1b346bdc767b service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:37.155979 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 806/1617] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:53:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:53:37.156891 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aee566-ce5e-490a-8228-115788260f4e service neutron] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:37.157985 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 812/1618] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:53:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:53:37.160015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7ef57-91d2-43fa-83d4-f673ccbd09db None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:37.160747 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7ef57-91d2-43fa-83d4-f673ccbd09db None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:37.161157 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7ef57-91d2-43fa-83d4-f673ccbd09db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:37.169394 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aee566-ce5e-490a-8228-115788260f4e service neutron] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:37.175892 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e976d8-ff34-4350-810b-b1087c7caeb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:37.176663 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e976d8-ff34-4350-810b-b1087c7caeb6 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:37.177040 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e976d8-ff34-4350-810b-b1087c7caeb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:37.177594 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e976d8-ff34-4350-810b-b1087c7caeb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:37.197548 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e976d8-ff34-4350-810b-b1087c7caeb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:37.200659 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e976d8-ff34-4350-810b-b1087c7caeb6 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:37.204247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-a2f7ef57-91d2-43fa-83d4-f673ccbd09db None None] MFA Rules not processed for user `5e69fd17e416401e84aa51537eed540f`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:37.205972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a2f7ef57-91d2-43fa-83d4-f673ccbd09db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:37.215669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 807/1619] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:53:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:53:37.217373 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e976d8-ff34-4350-810b-b1087c7caeb6 service neutron] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:37.231292 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:37.248261 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e976d8-ff34-4350-810b-b1087c7caeb6 service neutron] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:37.256927 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442cc75-f724-4c41-94db-9d41105dca80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:37.258016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442cc75-f724-4c41-94db-9d41105dca80 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:37.258016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442cc75-f724-4c41-94db-9d41105dca80 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:37.258715 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442cc75-f724-4c41-94db-9d41105dca80 admin admin] PATH_INFO: `/v3/projects/b32342e98f1e4d6d8279e7a7d019c0fa/users/6e1aff1e456a4396a7c8a76141ed6324/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:37.261470 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442cc75-f724-4c41-94db-9d41105dca80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b32342e98f1e4d6d8279e7a7d019c0fa, user_id=6e1aff1e456a4396a7c8a76141ed6324, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:37.266430 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442cc75-f724-4c41-94db-9d41105dca80 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:37.291047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 808/1620] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:53:37 2022] PUT /identity/v3/projects/b32342e98f1e4d6d8279e7a7d019c0fa/users/6e1aff1e456a4396a7c8a76141ed6324/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:53:37.293813 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854eec2f-df04-4783-a68f-d2c3f01284b6 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:37.294095 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854eec2f-df04-4783-a68f-d2c3f01284b6 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:37.294369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854eec2f-df04-4783-a68f-d2c3f01284b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:37.341269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 813/1621] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:37.347101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff508d32-cd9b-4e58-a151-9c6bd5fa8b97 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:37.347387 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff508d32-cd9b-4e58-a151-9c6bd5fa8b97 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:37.347664 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff508d32-cd9b-4e58-a151-9c6bd5fa8b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:37.359422 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-854eec2f-df04-4783-a68f-d2c3f01284b6 None None] MFA Rules not processed for user `e68a4a654a1e4a49b9e59cff12b68e72`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:37.361873 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-854eec2f-df04-4783-a68f-d2c3f01284b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:37.387286 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-ff508d32-cd9b-4e58-a151-9c6bd5fa8b97 None None] MFA Rules not processed for user `6e1aff1e456a4396a7c8a76141ed6324`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:37.389312 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-ff508d32-cd9b-4e58-a151-9c6bd5fa8b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:37.457007 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 809/1622] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:37.459085 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e3936-7c40-46b2-9927-6f48562e4e9a None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:37.459480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e3936-7c40-46b2-9927-6f48562e4e9a None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:37.459765 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e3936-7c40-46b2-9927-6f48562e4e9a None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:37.462795 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 810/1623] 10.222.0.59 () {54 vars in 962 bytes} [Fri Aug 26 00:53:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:53:37.463836 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e3936-7c40-46b2-9927-6f48562e4e9a None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:37.521693 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 814/1624] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:37 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:37.528878 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b4ad1-d7e5-4166-8671-e751350f0577 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:37.528978 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b4ad1-d7e5-4166-8671-e751350f0577 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:37.529299 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b4ad1-d7e5-4166-8671-e751350f0577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:37.597616 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-107b4ad1-d7e5-4166-8671-e751350f0577 None None] MFA Rules not processed for user `0602593263ca4b05a657d5b7697f932f`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:37.598901 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-107b4ad1-d7e5-4166-8671-e751350f0577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:37.600584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e3936-7c40-46b2-9927-6f48562e4e9a None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:37.608969 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6967d04-2e2c-4ba0-961b-4762ef252184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:37.609999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6967d04-2e2c-4ba0-961b-4762ef252184 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:37.611272 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6967d04-2e2c-4ba0-961b-4762ef252184 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:37.611705 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6967d04-2e2c-4ba0-961b-4762ef252184 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:37.613409 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6967d04-2e2c-4ba0-961b-4762ef252184 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:37.617865 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6967d04-2e2c-4ba0-961b-4762ef252184 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:37.708387 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 815/1625] 10.222.0.59 () {58 vars in 1059 bytes} [Fri Aug 26 00:53:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:37.710117 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b4ad1-d7e5-4166-8671-e751350f0577 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:37.712249 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 811/1626] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:53:37 2022] POST /identity/v3/projects => generated 371 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:53:37.717125 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6967d04-2e2c-4ba0-961b-4762ef252184 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:37.737578 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:37.738394 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:37.792301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b4ad1-d7e5-4166-8671-e751350f0577 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:37.815251 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6967d04-2e2c-4ba0-961b-4762ef252184 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:37.822048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da81ad7-600f-4ec0-b30c-9110de852d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:37.822743 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da81ad7-600f-4ec0-b30c-9110de852d84 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:37.823028 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da81ad7-600f-4ec0-b30c-9110de852d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:37.823304 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da81ad7-600f-4ec0-b30c-9110de852d84 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:37.823945 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da81ad7-600f-4ec0-b30c-9110de852d84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:37.825933 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da81ad7-600f-4ec0-b30c-9110de852d84 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:37.878541 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e1f85-517e-47e3-a477-8c6343612142 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:37.878541 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e1f85-517e-47e3-a477-8c6343612142 service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:37.878541 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e1f85-517e-47e3-a477-8c6343612142 service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:37.879414 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e1f85-517e-47e3-a477-8c6343612142 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:37.921098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 812/1627] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:53:37 2022] POST /identity/v3/users => generated 292 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:53:37.921098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da81ad7-600f-4ec0-b30c-9110de852d84 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:37.974413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e1f85-517e-47e3-a477-8c6343612142 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:37.977553 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e1f85-517e-47e3-a477-8c6343612142 service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:38.010952 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 816/1628] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:53:37 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:53:38.012159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e1f85-517e-47e3-a477-8c6343612142 service cinder] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:38.041502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e1f85-517e-47e3-a477-8c6343612142 service cinder] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:38.049904 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a266053-a51d-4230-b2b6-31bcdf8a1200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:38.050753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a266053-a51d-4230-b2b6-31bcdf8a1200 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:38.051109 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a266053-a51d-4230-b2b6-31bcdf8a1200 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:38.051450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a266053-a51d-4230-b2b6-31bcdf8a1200 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:38.052178 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a266053-a51d-4230-b2b6-31bcdf8a1200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:38.054828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a266053-a51d-4230-b2b6-31bcdf8a1200 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:38.057236 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da81ad7-600f-4ec0-b30c-9110de852d84 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:38.067738 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 817/1629] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:53:38 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:53:38.070346 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d6fde-2c3d-48e9-a3dd-993c16df81aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:38.071114 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d6fde-2c3d-48e9-a3dd-993c16df81aa service nova] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:38.071396 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d6fde-2c3d-48e9-a3dd-993c16df81aa service nova] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:38.071664 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d6fde-2c3d-48e9-a3dd-993c16df81aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:38.072515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a266053-a51d-4230-b2b6-31bcdf8a1200 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:38.087306 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a266053-a51d-4230-b2b6-31bcdf8a1200 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:38.092528 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d0932-346b-490f-8233-ee06ef9debdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:38.093219 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d0932-346b-490f-8233-ee06ef9debdd admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:38.093498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d0932-346b-490f-8233-ee06ef9debdd admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:38.093776 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d0932-346b-490f-8233-ee06ef9debdd admin admin] PATH_INFO: `/v3/projects/ebf050447dd84b188ae25db1fc470aa0/users/032ed13033534f178a06f6b361915f29/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:38.112289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d0932-346b-490f-8233-ee06ef9debdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebf050447dd84b188ae25db1fc470aa0, user_id=032ed13033534f178a06f6b361915f29, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:38.116722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d0932-346b-490f-8233-ee06ef9debdd admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:38.148876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d6fde-2c3d-48e9-a3dd-993c16df81aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:38.150902 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d6fde-2c3d-48e9-a3dd-993c16df81aa service nova] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:38.189281 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 813/1630] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:53:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:53:38.285066 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 818/1631] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:53:38 2022] PUT /identity/v3/projects/ebf050447dd84b188ae25db1fc470aa0/users/032ed13033534f178a06f6b361915f29/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:53:38.289658 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d6fde-2c3d-48e9-a3dd-993c16df81aa service nova] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:38.309698 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d6fde-2c3d-48e9-a3dd-993c16df81aa service nova] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:38.434451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:38.462926 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d0932-346b-490f-8233-ee06ef9debdd admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:38.517904 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b42a4f-fbc8-4f4e-a934-0f714f988067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:38.520837 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b42a4f-fbc8-4f4e-a934-0f714f988067 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:38.520837 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b42a4f-fbc8-4f4e-a934-0f714f988067 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:38.522212 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b42a4f-fbc8-4f4e-a934-0f714f988067 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:38.522942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b42a4f-fbc8-4f4e-a934-0f714f988067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:38.526865 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b42a4f-fbc8-4f4e-a934-0f714f988067 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:38.550206 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 814/1632] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:53:38 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:53:38.558754 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b42a4f-fbc8-4f4e-a934-0f714f988067 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:38.581726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b42a4f-fbc8-4f4e-a934-0f714f988067 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:38.592542 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f275c60-f527-480c-831e-e710712eda88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:38.593419 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f275c60-f527-480c-831e-e710712eda88 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:38.593639 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f275c60-f527-480c-831e-e710712eda88 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:38.593919 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f275c60-f527-480c-831e-e710712eda88 admin admin] PATH_INFO: `/v3/projects/ebf050447dd84b188ae25db1fc470aa0/users/032ed13033534f178a06f6b361915f29/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:38.596499 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d0932-346b-490f-8233-ee06ef9debdd admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:38.600833 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f275c60-f527-480c-831e-e710712eda88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebf050447dd84b188ae25db1fc470aa0, user_id=032ed13033534f178a06f6b361915f29, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:38.607107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f275c60-f527-480c-831e-e710712eda88 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:38.611158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2eff9-8d6e-4a0f-a568-e5ddf58dd735 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:38.611158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2eff9-8d6e-4a0f-a568-e5ddf58dd735 service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:38.611158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2eff9-8d6e-4a0f-a568-e5ddf58dd735 service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:38.611158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2eff9-8d6e-4a0f-a568-e5ddf58dd735 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:38.620675 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:38.639795 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 815/1633] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:53:38 2022] PUT /identity/v3/projects/ebf050447dd84b188ae25db1fc470aa0/users/032ed13033534f178a06f6b361915f29/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:53:38.650153 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4d812-0802-4f1d-b83a-ec253fba58d8 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:38.650475 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4d812-0802-4f1d-b83a-ec253fba58d8 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:38.650744 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4d812-0802-4f1d-b83a-ec253fba58d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:38.726786 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-0fe4d812-0802-4f1d-b83a-ec253fba58d8 None None] MFA Rules not processed for user `032ed13033534f178a06f6b361915f29`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:38.728941 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-0fe4d812-0802-4f1d-b83a-ec253fba58d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:38.729800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2eff9-8d6e-4a0f-a568-e5ddf58dd735 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:38.731782 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2eff9-8d6e-4a0f-a568-e5ddf58dd735 service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:38.746234 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 819/1634] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:53:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:53:38.862944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 816/1635] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:38 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:53:38.884731 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2eff9-8d6e-4a0f-a568-e5ddf58dd735 service cinder] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:38.903956 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2eff9-8d6e-4a0f-a568-e5ddf58dd735 service cinder] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:38.959351 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e209c-a1cf-4ef5-9da5-438482361a6b None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:38.960579 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e209c-a1cf-4ef5-9da5-438482361a6b None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:38.961396 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e209c-a1cf-4ef5-9da5-438482361a6b None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:38.963736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 817/1636] 10.222.0.59 () {54 vars in 962 bytes} [Fri Aug 26 00:53:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:53:38.972864 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a387eb-cff3-45e0-8d8f-04d12cbf1107 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:38.973152 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a387eb-cff3-45e0-8d8f-04d12cbf1107 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:38.973505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a387eb-cff3-45e0-8d8f-04d12cbf1107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:39.004940 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-79a387eb-cff3-45e0-8d8f-04d12cbf1107 None None] MFA Rules not processed for user `0602593263ca4b05a657d5b7697f932f`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:39.006369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-79a387eb-cff3-45e0-8d8f-04d12cbf1107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:39.059308 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5a2d6-a899-480c-ba76-be00eb49cf5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:39.061414 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5a2d6-a899-480c-ba76-be00eb49cf5f service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:39.062681 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5a2d6-a899-480c-ba76-be00eb49cf5f service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:39.063233 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5a2d6-a899-480c-ba76-be00eb49cf5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:39.084775 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a5a2d6-a899-480c-ba76-be00eb49cf5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:39.087740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a5a2d6-a899-480c-ba76-be00eb49cf5f service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:39.101423 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 818/1637] 10.222.0.59 () {58 vars in 1059 bytes} [Fri Aug 26 00:53:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:39.106563 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 820/1638] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:53:38 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:53:39.110960 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a11d69-b6a0-4eec-a158-c0a0882f639c None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:39.111465 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a11d69-b6a0-4eec-a158-c0a0882f639c None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:39.111958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a11d69-b6a0-4eec-a158-c0a0882f639c None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:39.115026 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 819/1639] 10.222.0.59 () {54 vars in 912 bytes} [Fri Aug 26 00:53:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:53:39.123981 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4884d0c-d423-4ab6-9a6b-d9b092ed7006 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:39.124568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4884d0c-d423-4ab6-9a6b-d9b092ed7006 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:39.125068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4884d0c-d423-4ab6-9a6b-d9b092ed7006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:39.162647 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-d4884d0c-d423-4ab6-9a6b-d9b092ed7006 None None] MFA Rules not processed for user `0602593263ca4b05a657d5b7697f932f`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:39.164985 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-d4884d0c-d423-4ab6-9a6b-d9b092ed7006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:39.180290 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 821/1640] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Aug 26 00:53:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:39.231505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a11d69-b6a0-4eec-a158-c0a0882f639c None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:39.244586 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a11d69-b6a0-4eec-a158-c0a0882f639c None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:39.248256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7fc1e-1fe2-42fc-9bac-b62284d7c187 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:39.250877 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7fc1e-1fe2-42fc-9bac-b62284d7c187 service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:39.250877 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7fc1e-1fe2-42fc-9bac-b62284d7c187 service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:39.250877 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7fc1e-1fe2-42fc-9bac-b62284d7c187 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:39.260443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a7fc1e-1fe2-42fc-9bac-b62284d7c187 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:39.261905 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a7fc1e-1fe2-42fc-9bac-b62284d7c187 service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:39.274406 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 820/1641] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:53:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:53:39.616936 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744c7ae-6efa-4eb3-a29f-9129e5a9fb48 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:39.616936 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744c7ae-6efa-4eb3-a29f-9129e5a9fb48 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:39.616936 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744c7ae-6efa-4eb3-a29f-9129e5a9fb48 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:39.616936 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 822/1642] 10.222.0.59 () {54 vars in 912 bytes} [Fri Aug 26 00:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:53:39.623775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f59ed1-5eb1-4d83-95f2-1a3290112b0f None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:39.623775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f59ed1-5eb1-4d83-95f2-1a3290112b0f None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:39.623775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f59ed1-5eb1-4d83-95f2-1a3290112b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:39.623775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-70f59ed1-5eb1-4d83-95f2-1a3290112b0f None None] MFA Rules not processed for user `0602593263ca4b05a657d5b7697f932f`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:39.623775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-70f59ed1-5eb1-4d83-95f2-1a3290112b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:39.623775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 821/1643] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Aug 26 00:53:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:53:39.625940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2744c7ae-6efa-4eb3-a29f-9129e5a9fb48 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:39.627564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f59ed1-5eb1-4d83-95f2-1a3290112b0f None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:39.628252 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2744c7ae-6efa-4eb3-a29f-9129e5a9fb48 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:39.628519 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f59ed1-5eb1-4d83-95f2-1a3290112b0f None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:39.628899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269fd43e-e1c9-444d-baa7-f717e733f5b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:39.631972 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d833aa8-6967-4201-bf50-60eae8f4012a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:39.632079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269fd43e-e1c9-444d-baa7-f717e733f5b7 service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:39.632158 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d833aa8-6967-4201-bf50-60eae8f4012a service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:39.632261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269fd43e-e1c9-444d-baa7-f717e733f5b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:39.632352 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d833aa8-6967-4201-bf50-60eae8f4012a service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:39.632499 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269fd43e-e1c9-444d-baa7-f717e733f5b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:39.632579 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d833aa8-6967-4201-bf50-60eae8f4012a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:39.632641 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269fd43e-e1c9-444d-baa7-f717e733f5b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:39.632641 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269fd43e-e1c9-444d-baa7-f717e733f5b7 service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:39.632825 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:39.632896 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 823/1644] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:53:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:53:39.673902 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d833aa8-6967-4201-bf50-60eae8f4012a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:39.675675 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d833aa8-6967-4201-bf50-60eae8f4012a service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:39.688644 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 822/1645] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:53:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:53:39.716769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269fd43e-e1c9-444d-baa7-f717e733f5b7 service cinder] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:39.806775 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269fd43e-e1c9-444d-baa7-f717e733f5b7 service cinder] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:39.812919 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d23431-c4df-4ec8-8800-3d0e8f902853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:39.813734 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d23431-c4df-4ec8-8800-3d0e8f902853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:39.814031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d23431-c4df-4ec8-8800-3d0e8f902853 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:39.814463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d23431-c4df-4ec8-8800-3d0e8f902853 admin admin] PATH_INFO: `/v3/users/5e69fd17e416401e84aa51537eed540f` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:39.816315 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d23431-c4df-4ec8-8800-3d0e8f902853 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e69fd17e416401e84aa51537eed540f)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:39.817961 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: WARNING py.warnings [None req-b7d23431-c4df-4ec8-8800-3d0e8f902853 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 00:53:39.817961 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: warnings.warn(msg) Aug 26 00:53:39.817961 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: Aug 26 00:53:39.824583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d23431-c4df-4ec8-8800-3d0e8f902853 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:40.070302 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-b7d23431-c4df-4ec8-8800-3d0e8f902853 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e69fd17e416401e84aa51537eed540f'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:40.071020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-b7d23431-c4df-4ec8-8800-3d0e8f902853 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e69fd17e416401e84aa51537eed540f'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:40.106294 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-b7d23431-c4df-4ec8-8800-3d0e8f902853 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e69fd17e416401e84aa51537eed540f'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:40.191223 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:40.191223 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-b7d23431-c4df-4ec8-8800-3d0e8f902853 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e69fd17e416401e84aa51537eed540f'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:40.192190 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-b7d23431-c4df-4ec8-8800-3d0e8f902853 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e69fd17e416401e84aa51537eed540f'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:40.237493 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03c970-fd07-454b-89cf-8fbb0ec3572b None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:40.237931 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03c970-fd07-454b-89cf-8fbb0ec3572b None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:40.238100 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03c970-fd07-454b-89cf-8fbb0ec3572b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:40.241430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 824/1646] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:53:39 2022] DELETE /identity/v3/users/5e69fd17e416401e84aa51537eed540f => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:53:40.257895 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-7c03c970-fd07-454b-89cf-8fbb0ec3572b None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:40.260547 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-7c03c970-fd07-454b-89cf-8fbb0ec3572b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:40.332760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 823/1647] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:40.368922 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d23431-c4df-4ec8-8800-3d0e8f902853 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:40.370302 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-b7d23431-c4df-4ec8-8800-3d0e8f902853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:40.445151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d23431-c4df-4ec8-8800-3d0e8f902853 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:40.448867 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed20ce-696f-4576-8885-d8e490b2c6cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:40.449572 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed20ce-696f-4576-8885-d8e490b2c6cb service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:40.449862 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed20ce-696f-4576-8885-d8e490b2c6cb service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:40.451994 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed20ce-696f-4576-8885-d8e490b2c6cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:40.462883 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed20ce-696f-4576-8885-d8e490b2c6cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:40.465594 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed20ce-696f-4576-8885-d8e490b2c6cb service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:40.479570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 825/1648] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:53:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:53:42.004251 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03c970-fd07-454b-89cf-8fbb0ec3572b None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:42.006202 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-7c03c970-fd07-454b-89cf-8fbb0ec3572b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:42.027771 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03c970-fd07-454b-89cf-8fbb0ec3572b None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:42.056549 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cff052-799e-4368-984f-5d83c78a50f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:42.057456 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cff052-799e-4368-984f-5d83c78a50f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:42.057939 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cff052-799e-4368-984f-5d83c78a50f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:42.058349 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cff052-799e-4368-984f-5d83c78a50f4 admin admin] PATH_INFO: `/v3/projects/bd196a920ed04c0bac853f0556b4233d` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:42.062030 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cff052-799e-4368-984f-5d83c78a50f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd196a920ed04c0bac853f0556b4233d)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:42.063276 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: WARNING py.warnings [None req-40cff052-799e-4368-984f-5d83c78a50f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 00:53:42.063276 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: warnings.warn(msg) Aug 26 00:53:42.063276 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: Aug 26 00:53:42.069569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cff052-799e-4368-984f-5d83c78a50f4 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:42.379594 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-40cff052-799e-4368-984f-5d83c78a50f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd196a920ed04c0bac853f0556b4233d', 'request_id': 'req-40cff052-799e-4368-984f-5d83c78a50f4'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:42.451499 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-40cff052-799e-4368-984f-5d83c78a50f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd196a920ed04c0bac853f0556b4233d', 'request_id': 'req-40cff052-799e-4368-984f-5d83c78a50f4'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:42.508526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-40cff052-799e-4368-984f-5d83c78a50f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd196a920ed04c0bac853f0556b4233d', 'request_id': 'req-40cff052-799e-4368-984f-5d83c78a50f4'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:42.559831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-40cff052-799e-4368-984f-5d83c78a50f4 admin admin] The token cache is being invalidate because project bd196a920ed04c0bac853f0556b4233d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:53:42.560329 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-40cff052-799e-4368-984f-5d83c78a50f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:42.561148 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-40cff052-799e-4368-984f-5d83c78a50f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:42.563507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 824/1649] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:53:42 2022] DELETE /identity/v3/projects/bd196a920ed04c0bac853f0556b4233d => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:53:42.571451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad46e1-b58a-480a-95d3-47730dc6cb43 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:42.571451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad46e1-b58a-480a-95d3-47730dc6cb43 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:42.571451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad46e1-b58a-480a-95d3-47730dc6cb43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:42.610992 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-64ad46e1-b58a-480a-95d3-47730dc6cb43 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:42.612233 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-64ad46e1-b58a-480a-95d3-47730dc6cb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:42.722475 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 826/1650] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:42.732833 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b541c69-44e2-4f6a-9d0e-3398fec248c4 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:42.735322 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b541c69-44e2-4f6a-9d0e-3398fec248c4 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:42.736768 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b541c69-44e2-4f6a-9d0e-3398fec248c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:42.801165 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-0b541c69-44e2-4f6a-9d0e-3398fec248c4 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:42.802474 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-0b541c69-44e2-4f6a-9d0e-3398fec248c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:42.826499 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 825/1651] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:42.827926 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad46e1-b58a-480a-95d3-47730dc6cb43 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:42.849075 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad46e1-b58a-480a-95d3-47730dc6cb43 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:42.860067 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764167b4-586f-4b80-be53-1b14b8785840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:42.860786 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764167b4-586f-4b80-be53-1b14b8785840 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:42.862220 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764167b4-586f-4b80-be53-1b14b8785840 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:42.862220 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764167b4-586f-4b80-be53-1b14b8785840 admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:42.862220 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764167b4-586f-4b80-be53-1b14b8785840 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:53:42.862220 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764167b4-586f-4b80-be53-1b14b8785840 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:42.865579 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764167b4-586f-4b80-be53-1b14b8785840 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:42.880928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 827/1652] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:53:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:53:42.884986 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b541c69-44e2-4f6a-9d0e-3398fec248c4 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:42.895533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b541c69-44e2-4f6a-9d0e-3398fec248c4 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:42.903124 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca792c-4587-45e4-91e8-250bcb8e0318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:42.903998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca792c-4587-45e4-91e8-250bcb8e0318 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:42.904341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca792c-4587-45e4-91e8-250bcb8e0318 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:42.904724 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca792c-4587-45e4-91e8-250bcb8e0318 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:42.905510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca792c-4587-45e4-91e8-250bcb8e0318 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:42.907746 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca792c-4587-45e4-91e8-250bcb8e0318 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:43.061036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 826/1653] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:53:42 2022] POST /identity/v3/projects => generated 403 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:53:43.065476 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764167b4-586f-4b80-be53-1b14b8785840 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:43.084305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764167b4-586f-4b80-be53-1b14b8785840 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:43.146709 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:43.206310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1fbd9-2bc2-4fe1-a9f6-11670c3faff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:43.207092 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1fbd9-2bc2-4fe1-a9f6-11670c3faff3 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:43.207473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1fbd9-2bc2-4fe1-a9f6-11670c3faff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:43.207817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1fbd9-2bc2-4fe1-a9f6-11670c3faff3 admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:43.208608 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b1fbd9-2bc2-4fe1-a9f6-11670c3faff3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:43.213450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b1fbd9-2bc2-4fe1-a9f6-11670c3faff3 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:43.360724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 828/1654] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:53:43 2022] POST /identity/v3/users => generated 302 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:53:43.362179 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca792c-4587-45e4-91e8-250bcb8e0318 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:43.393836 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca792c-4587-45e4-91e8-250bcb8e0318 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:43.444725 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252b3cef-d4d8-4c44-9b62-9b3b506c1e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:43.446545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b3cef-d4d8-4c44-9b62-9b3b506c1e4e admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:43.447638 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b3cef-d4d8-4c44-9b62-9b3b506c1e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:43.449227 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b3cef-d4d8-4c44-9b62-9b3b506c1e4e admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:43.451856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252b3cef-d4d8-4c44-9b62-9b3b506c1e4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:43.456352 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252b3cef-d4d8-4c44-9b62-9b3b506c1e4e admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:43.475053 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:43.487158 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 827/1655] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:53:43.491943 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1fbd9-2bc2-4fe1-a9f6-11670c3faff3 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:43.515320 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1fbd9-2bc2-4fe1-a9f6-11670c3faff3 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:43.548411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c4d0b-b783-42aa-9dba-75948f235e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:43.549616 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c4d0b-b783-42aa-9dba-75948f235e16 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:43.551221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c4d0b-b783-42aa-9dba-75948f235e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:43.554207 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c4d0b-b783-42aa-9dba-75948f235e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:43.559448 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c4d0b-b783-42aa-9dba-75948f235e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:43.563837 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c4d0b-b783-42aa-9dba-75948f235e16 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:43.599669 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 829/1656] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:53:43.600335 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252b3cef-d4d8-4c44-9b62-9b3b506c1e4e admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:43.611898 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252b3cef-d4d8-4c44-9b62-9b3b506c1e4e admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:43.628861 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bed986-0a32-4f49-b626-64e2d6a0cd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:43.629568 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bed986-0a32-4f49-b626-64e2d6a0cd10 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:43.629927 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bed986-0a32-4f49-b626-64e2d6a0cd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:43.630265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bed986-0a32-4f49-b626-64e2d6a0cd10 admin admin] PATH_INFO: `/v3/projects/048f09e433c343ba87faa257b37184c6/users/1306607a76a1432993994adda4d87358/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:43.699772 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bed986-0a32-4f49-b626-64e2d6a0cd10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=048f09e433c343ba87faa257b37184c6, user_id=1306607a76a1432993994adda4d87358, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:43.703479 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bed986-0a32-4f49-b626-64e2d6a0cd10 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:43.803608 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 828/1657] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:53:43 2022] PUT /identity/v3/projects/048f09e433c343ba87faa257b37184c6/users/1306607a76a1432993994adda4d87358/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:53:43.808989 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f7cf1-eaf9-45b1-9cb3-09034107d417 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:43.809637 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f7cf1-eaf9-45b1-9cb3-09034107d417 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:43.810106 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f7cf1-eaf9-45b1-9cb3-09034107d417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:43.872582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-fa2f7cf1-eaf9-45b1-9cb3-09034107d417 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:43.874789 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-fa2f7cf1-eaf9-45b1-9cb3-09034107d417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:44.113158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 830/1658] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:44.125203 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bed986-0a32-4f49-b626-64e2d6a0cd10 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:44.127748 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-26bed986-0a32-4f49-b626-64e2d6a0cd10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:44.269512 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bed986-0a32-4f49-b626-64e2d6a0cd10 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:44.279610 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad82d12e-d281-4e1b-9aca-f2a79c04e98d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:44.280661 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82d12e-d281-4e1b-9aca-f2a79c04e98d service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:44.281005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82d12e-d281-4e1b-9aca-f2a79c04e98d service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:44.281612 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82d12e-d281-4e1b-9aca-f2a79c04e98d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:44.322756 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad82d12e-d281-4e1b-9aca-f2a79c04e98d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:44.324502 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad82d12e-d281-4e1b-9aca-f2a79c04e98d service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:44.353556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 829/1659] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:53:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:53:47.892041 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd5461-9ab5-4db9-903a-890d84463fc2 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:47.893049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd5461-9ab5-4db9-903a-890d84463fc2 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:47.893552 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd5461-9ab5-4db9-903a-890d84463fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:47.956269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-66dd5461-9ab5-4db9-903a-890d84463fc2 None None] MFA Rules not processed for user `1306607a76a1432993994adda4d87358`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:47.961164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-66dd5461-9ab5-4db9-903a-890d84463fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:48.125011 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 831/1660] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:47 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:48.130109 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad82d12e-d281-4e1b-9aca-f2a79c04e98d service neutron] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:48.142406 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad82d12e-d281-4e1b-9aca-f2a79c04e98d service neutron] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:48.152736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2af87c-e375-4c0f-a99d-75884d07378c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:48.153444 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2af87c-e375-4c0f-a99d-75884d07378c admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:48.153621 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2af87c-e375-4c0f-a99d-75884d07378c admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:48.153858 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2af87c-e375-4c0f-a99d-75884d07378c admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:48.158171 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2af87c-e375-4c0f-a99d-75884d07378c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:48.166439 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2af87c-e375-4c0f-a99d-75884d07378c admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:48.252279 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 830/1661] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:53:48 2022] POST /identity/v3/projects => generated 405 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:53:48.256765 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd5461-9ab5-4db9-903a-890d84463fc2 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:48.277885 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd5461-9ab5-4db9-903a-890d84463fc2 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:48.340800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:48.367889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c2864e-783c-4c2d-b980-b38f3c23f0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:48.369327 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2864e-783c-4c2d-b980-b38f3c23f0ce admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:48.369977 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2864e-783c-4c2d-b980-b38f3c23f0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:48.370452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2864e-783c-4c2d-b980-b38f3c23f0ce admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:48.371297 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c2864e-783c-4c2d-b980-b38f3c23f0ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:48.374019 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c2864e-783c-4c2d-b980-b38f3c23f0ce admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:48.586435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2af87c-e375-4c0f-a99d-75884d07378c admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:48.589845 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 832/1662] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:53:48 2022] POST /identity/v3/users => generated 309 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:53:48.616803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2af87c-e375-4c0f-a99d-75884d07378c admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:48.649814 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29265d-23c0-4065-888b-a00dd9af1a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:48.650745 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29265d-23c0-4065-888b-a00dd9af1a66 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:48.652905 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29265d-23c0-4065-888b-a00dd9af1a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:48.653373 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29265d-23c0-4065-888b-a00dd9af1a66 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:48.657275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29265d-23c0-4065-888b-a00dd9af1a66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:48.665110 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29265d-23c0-4065-888b-a00dd9af1a66 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:48.717867 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c2864e-783c-4c2d-b980-b38f3c23f0ce admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:48.719090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 831/1663] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:53:48 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:53:48.740950 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c2864e-783c-4c2d-b980-b38f3c23f0ce admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:48.762618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19851d4e-1afb-49c6-82e5-1080a3d5d8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:48.763416 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19851d4e-1afb-49c6-82e5-1080a3d5d8e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:48.763652 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19851d4e-1afb-49c6-82e5-1080a3d5d8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:48.764174 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19851d4e-1afb-49c6-82e5-1080a3d5d8e7 admin admin] PATH_INFO: `/v3/projects/ca1a99f0dbec4b7bab2d2078c36cb080/users/70bf957116dd41c39b51b57cd5377cc6/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:48.826158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19851d4e-1afb-49c6-82e5-1080a3d5d8e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca1a99f0dbec4b7bab2d2078c36cb080, user_id=70bf957116dd41c39b51b57cd5377cc6, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:48.830204 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19851d4e-1afb-49c6-82e5-1080a3d5d8e7 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:48.900903 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 833/1664] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:53:48 2022] PUT /identity/v3/projects/ca1a99f0dbec4b7bab2d2078c36cb080/users/70bf957116dd41c39b51b57cd5377cc6/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:53:48.905557 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29265d-23c0-4065-888b-a00dd9af1a66 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:48.921915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29265d-23c0-4065-888b-a00dd9af1a66 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:49.049944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437fd883-9a3f-4962-86a2-9d9e29b7aba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:49.051903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fd883-9a3f-4962-86a2-9d9e29b7aba7 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:49.053435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fd883-9a3f-4962-86a2-9d9e29b7aba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:49.055477 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fd883-9a3f-4962-86a2-9d9e29b7aba7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:49.057719 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437fd883-9a3f-4962-86a2-9d9e29b7aba7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:49.062401 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437fd883-9a3f-4962-86a2-9d9e29b7aba7 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:49.093901 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 832/1665] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:53:48 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:53:49.099532 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19851d4e-1afb-49c6-82e5-1080a3d5d8e7 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:49.119965 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19851d4e-1afb-49c6-82e5-1080a3d5d8e7 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:49.125607 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250b6f4-f8c1-4185-b0e8-a29d2811b059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:49.126289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250b6f4-f8c1-4185-b0e8-a29d2811b059 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:49.126568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250b6f4-f8c1-4185-b0e8-a29d2811b059 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:49.126841 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250b6f4-f8c1-4185-b0e8-a29d2811b059 admin admin] PATH_INFO: `/v3/projects/ca1a99f0dbec4b7bab2d2078c36cb080/users/70bf957116dd41c39b51b57cd5377cc6/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:49.140968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e250b6f4-f8c1-4185-b0e8-a29d2811b059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca1a99f0dbec4b7bab2d2078c36cb080, user_id=70bf957116dd41c39b51b57cd5377cc6, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:49.156492 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e250b6f4-f8c1-4185-b0e8-a29d2811b059 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:49.180529 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 834/1666] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:53:49 2022] PUT /identity/v3/projects/ca1a99f0dbec4b7bab2d2078c36cb080/users/70bf957116dd41c39b51b57cd5377cc6/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:53:50.254584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445179b1-222d-4135-a299-d92d53c23eb3 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:50.254584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445179b1-222d-4135-a299-d92d53c23eb3 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:50.255089 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445179b1-222d-4135-a299-d92d53c23eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:50.350031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-445179b1-222d-4135-a299-d92d53c23eb3 None None] MFA Rules not processed for user `0cc36def73aa4bedb9e79d4cf11f5e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:50.352420 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-445179b1-222d-4135-a299-d92d53c23eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:50.479161 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:50.570778 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 833/1667] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:50.581349 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ce88a-7929-411d-8f7b-6ceaa124c4df None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:50.581808 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ce88a-7929-411d-8f7b-6ceaa124c4df None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:50.581916 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ce88a-7929-411d-8f7b-6ceaa124c4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:50.649473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-ed4ce88a-7929-411d-8f7b-6ceaa124c4df None None] MFA Rules not processed for user `0cc36def73aa4bedb9e79d4cf11f5e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:50.652882 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-ed4ce88a-7929-411d-8f7b-6ceaa124c4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:50.682247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 835/1668] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:50.722861 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445179b1-222d-4135-a299-d92d53c23eb3 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:50.725195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-445179b1-222d-4135-a299-d92d53c23eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:50.944081 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445179b1-222d-4135-a299-d92d53c23eb3 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:50.971483 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fca53f-0c50-4c9d-bbd6-ca1cdbd13bf7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:50.973192 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fca53f-0c50-4c9d-bbd6-ca1cdbd13bf7 service nova] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:50.973192 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fca53f-0c50-4c9d-bbd6-ca1cdbd13bf7 service nova] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:50.973192 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fca53f-0c50-4c9d-bbd6-ca1cdbd13bf7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:51.033213 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fca53f-0c50-4c9d-bbd6-ca1cdbd13bf7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:51.038952 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fca53f-0c50-4c9d-bbd6-ca1cdbd13bf7 service nova] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:51.061054 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 834/1669] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:53:50 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:53:51.350053 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536128cb-5418-4ce2-95ad-5a696307488f None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:51.364257 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536128cb-5418-4ce2-95ad-5a696307488f None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:51.364257 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536128cb-5418-4ce2-95ad-5a696307488f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:51.402771 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-536128cb-5418-4ce2-95ad-5a696307488f None None] MFA Rules not processed for user `1a425027e59242bba6b05407d19cac82`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:51.404223 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-536128cb-5418-4ce2-95ad-5a696307488f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:51.575464 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 836/1670] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:51.577314 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67100c55-b50e-48cd-b7b5-de2386c33bd0 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:51.584717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67100c55-b50e-48cd-b7b5-de2386c33bd0 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:51.585031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67100c55-b50e-48cd-b7b5-de2386c33bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:51.623157 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-67100c55-b50e-48cd-b7b5-de2386c33bd0 None None] MFA Rules not processed for user `1a425027e59242bba6b05407d19cac82`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:51.626024 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-67100c55-b50e-48cd-b7b5-de2386c33bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:51.651182 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 835/1671] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:51.663046 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536128cb-5418-4ce2-95ad-5a696307488f None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:51.664871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-536128cb-5418-4ce2-95ad-5a696307488f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:51.690034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536128cb-5418-4ce2-95ad-5a696307488f None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:51.701950 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb9c32-7844-4c74-8ed5-bc7465dc132d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:51.703269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb9c32-7844-4c74-8ed5-bc7465dc132d service nova] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:51.703726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb9c32-7844-4c74-8ed5-bc7465dc132d service nova] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:51.710540 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb9c32-7844-4c74-8ed5-bc7465dc132d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:51.723755 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb9c32-7844-4c74-8ed5-bc7465dc132d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:51.731411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb9c32-7844-4c74-8ed5-bc7465dc132d service nova] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:51.754289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 837/1672] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:53:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:53:52.334568 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d7fd6-41c5-4868-8c51-ddf7f532a79b None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:52.335002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d7fd6-41c5-4868-8c51-ddf7f532a79b None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:52.335521 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d7fd6-41c5-4868-8c51-ddf7f532a79b None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:52.337150 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 836/1673] 10.222.0.59 () {54 vars in 912 bytes} [Fri Aug 26 00:53:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:53:52.343945 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a10c1e-e2f9-49b2-a9fb-172964108d8e None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:52.344235 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a10c1e-e2f9-49b2-a9fb-172964108d8e None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:52.344563 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a10c1e-e2f9-49b2-a9fb-172964108d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:52.439020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-07a10c1e-e2f9-49b2-a9fb-172964108d8e None None] MFA Rules not processed for user `4698a49ed4a04f58a488359a2742e5da`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:52.443049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-07a10c1e-e2f9-49b2-a9fb-172964108d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:52.558515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 838/1674] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Aug 26 00:53:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:52.674738 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d7fd6-41c5-4868-8c51-ddf7f532a79b None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:52.679183 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-8f9d7fd6-41c5-4868-8c51-ddf7f532a79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:52.703958 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa475abf-3f20-45e3-9e83-4de85d45cf38 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:52.704434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa475abf-3f20-45e3-9e83-4de85d45cf38 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:52.704531 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa475abf-3f20-45e3-9e83-4de85d45cf38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:52.729817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:52.765508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d7fd6-41c5-4868-8c51-ddf7f532a79b None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:52.770163 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3734ed-ca36-4e46-9fe5-b931b69fe39a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:52.771168 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3734ed-ca36-4e46-9fe5-b931b69fe39a service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:52.771792 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3734ed-ca36-4e46-9fe5-b931b69fe39a service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:52.773607 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3734ed-ca36-4e46-9fe5-b931b69fe39a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:52.789179 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3734ed-ca36-4e46-9fe5-b931b69fe39a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:52.792138 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3734ed-ca36-4e46-9fe5-b931b69fe39a service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:52.804433 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-fa475abf-3f20-45e3-9e83-4de85d45cf38 None None] MFA Rules not processed for user `70bf957116dd41c39b51b57cd5377cc6`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:52.805909 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-fa475abf-3f20-45e3-9e83-4de85d45cf38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:52.829723 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 837/1675] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:53:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:53:52.835293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931866b3-f273-4eb4-a3d3-dc9dbec0a4c2 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:52.836483 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931866b3-f273-4eb4-a3d3-dc9dbec0a4c2 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:52.837049 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931866b3-f273-4eb4-a3d3-dc9dbec0a4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:52.929654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-931866b3-f273-4eb4-a3d3-dc9dbec0a4c2 None None] MFA Rules not processed for user `34240c306dc54f0c8758dbb4b4525376`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:52.942777 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-931866b3-f273-4eb4-a3d3-dc9dbec0a4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:53.074255 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 839/1676] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:52 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:53:53.075986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079a3e6-08da-4597-bacf-a9698d62209b None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:53.076422 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079a3e6-08da-4597-bacf-a9698d62209b None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:53.076846 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079a3e6-08da-4597-bacf-a9698d62209b None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:53.079058 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 840/1677] 10.222.0.59 () {54 vars in 912 bytes} [Fri Aug 26 00:53:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:53:53.085344 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae445a5-1835-401b-a57f-1426167abe9a None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:53.085844 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae445a5-1835-401b-a57f-1426167abe9a None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:53.087445 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae445a5-1835-401b-a57f-1426167abe9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:53.227026 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-5ae445a5-1835-401b-a57f-1426167abe9a None None] MFA Rules not processed for user `4698a49ed4a04f58a488359a2742e5da`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:53.231913 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-5ae445a5-1835-401b-a57f-1426167abe9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:53.270066 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 841/1678] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Aug 26 00:53:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:53.290879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 838/1679] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 451 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:53.301282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae445a5-1835-401b-a57f-1426167abe9a None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:53.312645 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae445a5-1835-401b-a57f-1426167abe9a None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:53.339034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1865f-2ace-413b-97b7-19d33da6b0ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:53.339897 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1865f-2ace-413b-97b7-19d33da6b0ae service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:53.342466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1865f-2ace-413b-97b7-19d33da6b0ae service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:53.342937 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1865f-2ace-413b-97b7-19d33da6b0ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:53.359785 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1865f-2ace-413b-97b7-19d33da6b0ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:53.365205 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1865f-2ace-413b-97b7-19d33da6b0ae service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:53.400002 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 842/1680] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:53:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:53:54.033611 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb38017-900a-4f43-be27-7541a11fc340 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:54.034825 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb38017-900a-4f43-be27-7541a11fc340 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:54.035245 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb38017-900a-4f43-be27-7541a11fc340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:54.125757 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-2bb38017-900a-4f43-be27-7541a11fc340 None None] MFA Rules not processed for user `0175a2b440704d51aa14607a2e20e36a`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:54.127271 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-2bb38017-900a-4f43-be27-7541a11fc340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:54.159997 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:54.406859 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 839/1681] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:54.682064 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1865f-2ace-413b-97b7-19d33da6b0ae service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:54.709534 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1865f-2ace-413b-97b7-19d33da6b0ae service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:54.732974 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8590e79-df58-45ff-b59f-379bb17bee80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:54.733661 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8590e79-df58-45ff-b59f-379bb17bee80 service nova] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:54.733932 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8590e79-df58-45ff-b59f-379bb17bee80 service nova] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:54.734408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8590e79-df58-45ff-b59f-379bb17bee80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:54.768884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8590e79-df58-45ff-b59f-379bb17bee80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:54.770848 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8590e79-df58-45ff-b59f-379bb17bee80 service nova] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:54.791662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 843/1682] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:53:54 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:53:54.823633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb38017-900a-4f43-be27-7541a11fc340 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:54.825365 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-2bb38017-900a-4f43-be27-7541a11fc340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:54.859509 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8590e79-df58-45ff-b59f-379bb17bee80 service nova] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:54.882909 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8590e79-df58-45ff-b59f-379bb17bee80 service nova] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:54.895512 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1c96b-1b92-41d5-a4b3-79b3d6435add service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:54.897579 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1c96b-1b92-41d5-a4b3-79b3d6435add service nova] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:54.897969 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1c96b-1b92-41d5-a4b3-79b3d6435add service nova] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:54.898346 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1c96b-1b92-41d5-a4b3-79b3d6435add service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:54.919569 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef1c96b-1b92-41d5-a4b3-79b3d6435add service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:54.921795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef1c96b-1b92-41d5-a4b3-79b3d6435add service nova] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:54.955466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 844/1683] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:53:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:53:54.972411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28806155-e7da-4365-8f52-4edc939b661b None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:54.982955 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28806155-e7da-4365-8f52-4edc939b661b None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:54.982955 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28806155-e7da-4365-8f52-4edc939b661b None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:54.982955 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 845/1684] 10.222.0.59 () {54 vars in 961 bytes} [Fri Aug 26 00:53:54 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:53:54.983559 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec2f02-a8d3-440e-9db2-f38dc79a74cb None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:54.995695 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec2f02-a8d3-440e-9db2-f38dc79a74cb None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:54.996704 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec2f02-a8d3-440e-9db2-f38dc79a74cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:55.005411 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb38017-900a-4f43-be27-7541a11fc340 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:55.032175 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-03ec2f02-a8d3-440e-9db2-f38dc79a74cb None None] MFA Rules not processed for user `2876f4182f0949039b862f053533fe85`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:55.038009 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d27ee-d90b-4d20-9b4e-ea730a1c5c4a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2876f4182f0949039b862f053533fe85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:55.038721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d27ee-d90b-4d20-9b4e-ea730a1c5c4a service placement] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:55.040402 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d27ee-d90b-4d20-9b4e-ea730a1c5c4a service placement] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:55.044721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d27ee-d90b-4d20-9b4e-ea730a1c5c4a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:55.049658 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-03ec2f02-a8d3-440e-9db2-f38dc79a74cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:55.053775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-241d27ee-d90b-4d20-9b4e-ea730a1c5c4a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:55.111921 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 846/1685] 10.222.0.59 () {58 vars in 1058 bytes} [Fri Aug 26 00:53:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:55.118880 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec2f02-a8d3-440e-9db2-f38dc79a74cb None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:55.144173 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec2f02-a8d3-440e-9db2-f38dc79a74cb None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:55.144897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d27ee-d90b-4d20-9b4e-ea730a1c5c4a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:55.148098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d27ee-d90b-4d20-9b4e-ea730a1c5c4a service placement] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:55.165067 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37900c4b-44cf-43c6-98ee-c67c9c279cc9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2876f4182f0949039b862f053533fe85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:55.166737 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37900c4b-44cf-43c6-98ee-c67c9c279cc9 service placement] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:55.167200 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37900c4b-44cf-43c6-98ee-c67c9c279cc9 service placement] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:55.168760 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37900c4b-44cf-43c6-98ee-c67c9c279cc9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:55.218741 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 840/1686] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:53:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:53:55.222599 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37900c4b-44cf-43c6-98ee-c67c9c279cc9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:55.225284 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37900c4b-44cf-43c6-98ee-c67c9c279cc9 service placement] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:55.242335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 847/1687] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:53:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:53:55.516416 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d27ee-d90b-4d20-9b4e-ea730a1c5c4a service placement] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:55.517965 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-241d27ee-d90b-4d20-9b4e-ea730a1c5c4a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:55.579434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2633b-087a-4529-9b16-8c3b8a91fb17 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:55.579716 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2633b-087a-4529-9b16-8c3b8a91fb17 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:55.579994 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2633b-087a-4529-9b16-8c3b8a91fb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:55.620229 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-ddd2633b-087a-4529-9b16-8c3b8a91fb17 None None] MFA Rules not processed for user `0cc36def73aa4bedb9e79d4cf11f5e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:55.624580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-ddd2633b-087a-4529-9b16-8c3b8a91fb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:55.663694 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 848/1688] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:55 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:53:55.669377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2633b-087a-4529-9b16-8c3b8a91fb17 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:55.675290 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-ddd2633b-087a-4529-9b16-8c3b8a91fb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:55.765240 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d27ee-d90b-4d20-9b4e-ea730a1c5c4a service placement] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:55.778189 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c1ce62-3b72-40c9-bb9d-2d4fc1ea1127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:55.778846 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1ce62-3b72-40c9-bb9d-2d4fc1ea1127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:55.779126 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1ce62-3b72-40c9-bb9d-2d4fc1ea1127 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:55.779401 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1ce62-3b72-40c9-bb9d-2d4fc1ea1127 admin admin] PATH_INFO: `/v3/users/877072c4f9d24f5d9cdcad35817101d8` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:55.781941 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c1ce62-3b72-40c9-bb9d-2d4fc1ea1127 admin admin] RBAC: Authorizing `identity:delete_user(user_id=877072c4f9d24f5d9cdcad35817101d8)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:55.784016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c1ce62-3b72-40c9-bb9d-2d4fc1ea1127 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:55.880033 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2633b-087a-4529-9b16-8c3b8a91fb17 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:55.893020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96821ae-6cbd-4e49-bc9c-a07e847d7e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:55.893020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96821ae-6cbd-4e49-bc9c-a07e847d7e79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:55.893020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96821ae-6cbd-4e49-bc9c-a07e847d7e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:55.893020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96821ae-6cbd-4e49-bc9c-a07e847d7e79 admin admin] PATH_INFO: `/v3/users/b390ac3085984efc909f44c7152576ff` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:55.899408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96821ae-6cbd-4e49-bc9c-a07e847d7e79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b390ac3085984efc909f44c7152576ff)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:55.908055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96821ae-6cbd-4e49-bc9c-a07e847d7e79 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:55.940390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:56.100604 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c2c1ce62-3b72-40c9-bb9d-2d4fc1ea1127 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '877072c4f9d24f5d9cdcad35817101d8'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:56.102367 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:56.138527 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c2c1ce62-3b72-40c9-bb9d-2d4fc1ea1127 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '877072c4f9d24f5d9cdcad35817101d8'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:56.250840 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f96821ae-6cbd-4e49-bc9c-a07e847d7e79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b390ac3085984efc909f44c7152576ff'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:56.251566 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f96821ae-6cbd-4e49-bc9c-a07e847d7e79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b390ac3085984efc909f44c7152576ff'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:56.284727 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c2c1ce62-3b72-40c9-bb9d-2d4fc1ea1127 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '877072c4f9d24f5d9cdcad35817101d8'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:56.286182 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c2c1ce62-3b72-40c9-bb9d-2d4fc1ea1127 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '877072c4f9d24f5d9cdcad35817101d8'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:56.296653 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c2c1ce62-3b72-40c9-bb9d-2d4fc1ea1127 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '877072c4f9d24f5d9cdcad35817101d8'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:56.296961 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f96821ae-6cbd-4e49-bc9c-a07e847d7e79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b390ac3085984efc909f44c7152576ff'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:56.334257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 841/1689] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:53:55 2022] DELETE /identity/v3/users/877072c4f9d24f5d9cdcad35817101d8 => generated 0 bytes in 818 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:53:56.335939 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f344c-6da5-4ad0-a873-0397fe0b2cce None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:56.336431 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f344c-6da5-4ad0-a873-0397fe0b2cce None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:56.336887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f344c-6da5-4ad0-a873-0397fe0b2cce None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:56.340401 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 842/1690] 10.222.0.59 () {54 vars in 921 bytes} [Fri Aug 26 00:53:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:53:56.342173 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91678be6-c332-4333-b405-06d14c300cb4 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:56.342629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91678be6-c332-4333-b405-06d14c300cb4 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:56.343070 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91678be6-c332-4333-b405-06d14c300cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:56.382710 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f96821ae-6cbd-4e49-bc9c-a07e847d7e79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b390ac3085984efc909f44c7152576ff'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:56.383736 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f96821ae-6cbd-4e49-bc9c-a07e847d7e79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b390ac3085984efc909f44c7152576ff'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:56.407605 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 849/1691] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:53:55 2022] DELETE /identity/v3/users/b390ac3085984efc909f44c7152576ff => generated 0 bytes in 743 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:53:56.409286 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b93b70-408f-44ce-a475-ad1ae6d0f048 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:56.409582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b93b70-408f-44ce-a475-ad1ae6d0f048 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:56.409903 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b93b70-408f-44ce-a475-ad1ae6d0f048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:56.413000 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-91678be6-c332-4333-b405-06d14c300cb4 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:56.414228 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-91678be6-c332-4333-b405-06d14c300cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:56.484307 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-c0b93b70-408f-44ce-a475-ad1ae6d0f048 None None] MFA Rules not processed for user `2876f4182f0949039b862f053533fe85`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:56.485886 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-c0b93b70-408f-44ce-a475-ad1ae6d0f048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:56.616939 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 843/1692] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:56.621037 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacb926-595c-4853-9b3f-3e3aa891e8fb None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:56.621562 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacb926-595c-4853-9b3f-3e3aa891e8fb None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:56.622018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacb926-595c-4853-9b3f-3e3aa891e8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:56.636170 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 850/1693] 10.222.0.59 () {58 vars in 1018 bytes} [Fri Aug 26 00:53:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:56.668072 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-deacb926-595c-4853-9b3f-3e3aa891e8fb None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:56.669790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-deacb926-595c-4853-9b3f-3e3aa891e8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:56.695108 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 844/1694] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:56.715852 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b93b70-408f-44ce-a475-ad1ae6d0f048 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:56.718510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-c0b93b70-408f-44ce-a475-ad1ae6d0f048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:56.828940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b93b70-408f-44ce-a475-ad1ae6d0f048 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:56.833114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e66fe-dcba-4f95-8a6b-367cc1043080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:56.834190 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e66fe-dcba-4f95-8a6b-367cc1043080 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:56.835016 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e66fe-dcba-4f95-8a6b-367cc1043080 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:56.835123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e66fe-dcba-4f95-8a6b-367cc1043080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:56.873393 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deacb926-595c-4853-9b3f-3e3aa891e8fb None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:56.876418 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e66fe-dcba-4f95-8a6b-367cc1043080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:56.877791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e66fe-dcba-4f95-8a6b-367cc1043080 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:56.881649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-deacb926-595c-4853-9b3f-3e3aa891e8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:56.891338 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 851/1695] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:53:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:53:56.893168 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e66fe-dcba-4f95-8a6b-367cc1043080 service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:56.895203 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-c00e66fe-dcba-4f95-8a6b-367cc1043080 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:56.908498 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deacb926-595c-4853-9b3f-3e3aa891e8fb None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:56.926823 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e66fe-dcba-4f95-8a6b-367cc1043080 service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:56.928554 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f0714f-77f2-40a5-bb30-0a3acc87ada6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:56.929211 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0714f-77f2-40a5-bb30-0a3acc87ada6 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:56.930328 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0714f-77f2-40a5-bb30-0a3acc87ada6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:56.930651 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0714f-77f2-40a5-bb30-0a3acc87ada6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:56.934890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f03e4-3ef8-4963-8790-8ad672ad0c4b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2876f4182f0949039b862f053533fe85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:56.936042 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f03e4-3ef8-4963-8790-8ad672ad0c4b service placement] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:56.936607 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f03e4-3ef8-4963-8790-8ad672ad0c4b service placement] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:56.937207 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f03e4-3ef8-4963-8790-8ad672ad0c4b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:56.950032 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54f03e4-3ef8-4963-8790-8ad672ad0c4b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:56.952884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54f03e4-3ef8-4963-8790-8ad672ad0c4b service placement] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:56.966265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 852/1696] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:53:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:53:56.982832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f0714f-77f2-40a5-bb30-0a3acc87ada6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:56.984329 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f0714f-77f2-40a5-bb30-0a3acc87ada6 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:57.018417 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 845/1697] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:53:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:53:58.164026 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f03e4-3ef8-4963-8790-8ad672ad0c4b service placement] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:58.165763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-c54f03e4-3ef8-4963-8790-8ad672ad0c4b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:58.221571 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f03e4-3ef8-4963-8790-8ad672ad0c4b service placement] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:58.236808 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636cc46e-0748-4c87-b033-7c0738aac4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:58.237256 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cc46e-0748-4c87-b033-7c0738aac4c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:58.237540 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cc46e-0748-4c87-b033-7c0738aac4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:58.237816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cc46e-0748-4c87-b033-7c0738aac4c0 admin admin] PATH_INFO: `/v3/projects/0527db336d294c969ec669c8aa56ac2f` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:58.245899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636cc46e-0748-4c87-b033-7c0738aac4c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0527db336d294c969ec669c8aa56ac2f)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:58.248130 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636cc46e-0748-4c87-b033-7c0738aac4c0 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:58.290051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:58.296943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f0714f-77f2-40a5-bb30-0a3acc87ada6 service neutron] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:58.296943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-49f0714f-77f2-40a5-bb30-0a3acc87ada6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:58.345622 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f0714f-77f2-40a5-bb30-0a3acc87ada6 service neutron] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:58.370338 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12a66c-699d-48b8-b6f2-11db4f0044d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:58.382932 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12a66c-699d-48b8-b6f2-11db4f0044d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:58.385602 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12a66c-699d-48b8-b6f2-11db4f0044d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:58.388119 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12a66c-699d-48b8-b6f2-11db4f0044d6 admin admin] PATH_INFO: `/v3/projects/b0540b60f8e04a0584625e34fbcafa50` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:58.392894 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12a66c-699d-48b8-b6f2-11db4f0044d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0540b60f8e04a0584625e34fbcafa50)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:58.399708 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12a66c-699d-48b8-b6f2-11db4f0044d6 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:58.496886 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-636cc46e-0748-4c87-b033-7c0738aac4c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0527db336d294c969ec669c8aa56ac2f', 'request_id': 'req-636cc46e-0748-4c87-b033-7c0738aac4c0'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:58.589199 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-bc12a66c-699d-48b8-b6f2-11db4f0044d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0540b60f8e04a0584625e34fbcafa50', 'request_id': 'req-bc12a66c-699d-48b8-b6f2-11db4f0044d6'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:58.607808 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-636cc46e-0748-4c87-b033-7c0738aac4c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0527db336d294c969ec669c8aa56ac2f', 'request_id': 'req-636cc46e-0748-4c87-b033-7c0738aac4c0'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:58.629752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-636cc46e-0748-4c87-b033-7c0738aac4c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0527db336d294c969ec669c8aa56ac2f', 'request_id': 'req-636cc46e-0748-4c87-b033-7c0738aac4c0'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:58.647677 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-636cc46e-0748-4c87-b033-7c0738aac4c0 admin admin] The token cache is being invalidate because project 0527db336d294c969ec669c8aa56ac2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:53:58.648186 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-636cc46e-0748-4c87-b033-7c0738aac4c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:58.649091 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-636cc46e-0748-4c87-b033-7c0738aac4c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:58.653352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 853/1698] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:53:58 2022] DELETE /identity/v3/projects/0527db336d294c969ec669c8aa56ac2f => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:53:58.655038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636cc46e-0748-4c87-b033-7c0738aac4c0 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:58.658199 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-636cc46e-0748-4c87-b033-7c0738aac4c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:58.728264 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-bc12a66c-699d-48b8-b6f2-11db4f0044d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0540b60f8e04a0584625e34fbcafa50', 'request_id': 'req-bc12a66c-699d-48b8-b6f2-11db4f0044d6'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:58.745135 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-bc12a66c-699d-48b8-b6f2-11db4f0044d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0540b60f8e04a0584625e34fbcafa50', 'request_id': 'req-bc12a66c-699d-48b8-b6f2-11db4f0044d6'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:58.746047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636cc46e-0748-4c87-b033-7c0738aac4c0 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:58.798404 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-bc12a66c-699d-48b8-b6f2-11db4f0044d6 admin admin] The token cache is being invalidate because project b0540b60f8e04a0584625e34fbcafa50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:53:58.827360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-bc12a66c-699d-48b8-b6f2-11db4f0044d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:58.828999 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6aca1-1164-49b2-85cd-d97bce68aed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:58.829650 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6aca1-1164-49b2-85cd-d97bce68aed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:58.829936 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6aca1-1164-49b2-85cd-d97bce68aed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:58.830259 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6aca1-1164-49b2-85cd-d97bce68aed6 admin admin] PATH_INFO: `/v3/users/1306607a76a1432993994adda4d87358` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:58.831009 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-bc12a66c-699d-48b8-b6f2-11db4f0044d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:58.831554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b6aca1-1164-49b2-85cd-d97bce68aed6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1306607a76a1432993994adda4d87358)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:58.834139 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b6aca1-1164-49b2-85cd-d97bce68aed6 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:58.849438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 846/1699] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:53:58 2022] DELETE /identity/v3/projects/b0540b60f8e04a0584625e34fbcafa50 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:53:58.856190 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12a66c-699d-48b8-b6f2-11db4f0044d6 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:58.859543 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-bc12a66c-699d-48b8-b6f2-11db4f0044d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:58.898126 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12a66c-699d-48b8-b6f2-11db4f0044d6 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:58.922477 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27446bce-bc4c-40bb-8f94-02f25ebc5eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:58.923572 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27446bce-bc4c-40bb-8f94-02f25ebc5eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:58.924029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27446bce-bc4c-40bb-8f94-02f25ebc5eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:58.924560 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27446bce-bc4c-40bb-8f94-02f25ebc5eb1 admin admin] PATH_INFO: `/v3/users/6e1aff1e456a4396a7c8a76141ed6324` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:58.926185 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27446bce-bc4c-40bb-8f94-02f25ebc5eb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e1aff1e456a4396a7c8a76141ed6324)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:58.935391 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27446bce-bc4c-40bb-8f94-02f25ebc5eb1 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:59.020614 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:53:59.065129 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-a7b6aca1-1164-49b2-85cd-d97bce68aed6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1306607a76a1432993994adda4d87358'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:59.066356 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-a7b6aca1-1164-49b2-85cd-d97bce68aed6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1306607a76a1432993994adda4d87358'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:59.099744 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-a7b6aca1-1164-49b2-85cd-d97bce68aed6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1306607a76a1432993994adda4d87358'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:59.190447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-a7b6aca1-1164-49b2-85cd-d97bce68aed6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1306607a76a1432993994adda4d87358'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:59.191096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-a7b6aca1-1164-49b2-85cd-d97bce68aed6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1306607a76a1432993994adda4d87358'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:59.221416 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 854/1700] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:53:58 2022] DELETE /identity/v3/users/1306607a76a1432993994adda4d87358 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:53:59.225374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-27446bce-bc4c-40bb-8f94-02f25ebc5eb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e1aff1e456a4396a7c8a76141ed6324'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:59.225532 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0463561-4a3c-4368-b565-bbf46a9b37d1 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:59.229623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0463561-4a3c-4368-b565-bbf46a9b37d1 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:59.229841 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0463561-4a3c-4368-b565-bbf46a9b37d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:59.247414 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-27446bce-bc4c-40bb-8f94-02f25ebc5eb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e1aff1e456a4396a7c8a76141ed6324'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:59.273450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-a0463561-4a3c-4368-b565-bbf46a9b37d1 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:59.274731 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a0463561-4a3c-4368-b565-bbf46a9b37d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:59.363948 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-27446bce-bc4c-40bb-8f94-02f25ebc5eb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e1aff1e456a4396a7c8a76141ed6324'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:59.369672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-27446bce-bc4c-40bb-8f94-02f25ebc5eb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e1aff1e456a4396a7c8a76141ed6324'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:59.369672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-27446bce-bc4c-40bb-8f94-02f25ebc5eb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e1aff1e456a4396a7c8a76141ed6324'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:53:59.397802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 847/1701] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:53:58 2022] DELETE /identity/v3/users/6e1aff1e456a4396a7c8a76141ed6324 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:53:59.418838 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 855/1702] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:59.427921 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb80ed4-75c3-4d89-bdfe-c753a8242d7d None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:59.428778 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb80ed4-75c3-4d89-bdfe-c753a8242d7d None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:59.429110 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb80ed4-75c3-4d89-bdfe-c753a8242d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:59.452261 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-bdb80ed4-75c3-4d89-bdfe-c753a8242d7d None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:53:59.453758 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-bdb80ed4-75c3-4d89-bdfe-c753a8242d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:53:59.566345 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 848/1703] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:53:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:53:59.571674 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0463561-4a3c-4368-b565-bbf46a9b37d1 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:59.587711 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0463561-4a3c-4368-b565-bbf46a9b37d1 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:59.625636 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1238b-6552-4fc5-ba79-9068c6091912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:59.626587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1238b-6552-4fc5-ba79-9068c6091912 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:59.627039 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1238b-6552-4fc5-ba79-9068c6091912 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:59.627455 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1238b-6552-4fc5-ba79-9068c6091912 admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:59.628259 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da1238b-6552-4fc5-ba79-9068c6091912 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:53:59.628782 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da1238b-6552-4fc5-ba79-9068c6091912 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:59.634569 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da1238b-6552-4fc5-ba79-9068c6091912 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:59.669804 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 856/1704] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:53:59.672229 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb80ed4-75c3-4d89-bdfe-c753a8242d7d None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:59.691356 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb80ed4-75c3-4d89-bdfe-c753a8242d7d None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:59.702505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b581f1-b576-42cf-8cc3-1beda5d14eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:53:59.703263 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b581f1-b576-42cf-8cc3-1beda5d14eff admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:53:59.703551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b581f1-b576-42cf-8cc3-1beda5d14eff admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:53:59.704250 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b581f1-b576-42cf-8cc3-1beda5d14eff admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:53:59.704944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b581f1-b576-42cf-8cc3-1beda5d14eff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:53:59.707226 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b581f1-b576-42cf-8cc3-1beda5d14eff admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:53:59.834492 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 849/1705] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:53:59 2022] POST /identity/v3/projects => generated 381 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:53:59.841431 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1238b-6552-4fc5-ba79-9068c6091912 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:59.858207 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1238b-6552-4fc5-ba79-9068c6091912 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:53:59.870169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b581f1-b576-42cf-8cc3-1beda5d14eff admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:53:59.878332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-51b581f1-b576-42cf-8cc3-1beda5d14eff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:00.026690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ddc8c-7621-4ab2-8a8d-931372d31b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:00.027336 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ddc8c-7621-4ab2-8a8d-931372d31b71 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:00.027619 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ddc8c-7621-4ab2-8a8d-931372d31b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:00.027890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ddc8c-7621-4ab2-8a8d-931372d31b71 admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:00.031059 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ddc8c-7621-4ab2-8a8d-931372d31b71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:00.033304 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ddc8c-7621-4ab2-8a8d-931372d31b71 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:00.051431 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b581f1-b576-42cf-8cc3-1beda5d14eff admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:00.056877 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838c6dc-e6aa-4f6a-ba3f-cfbaec50351e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:00.057669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838c6dc-e6aa-4f6a-ba3f-cfbaec50351e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:00.058012 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838c6dc-e6aa-4f6a-ba3f-cfbaec50351e admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:00.058346 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838c6dc-e6aa-4f6a-ba3f-cfbaec50351e admin admin] PATH_INFO: `/v3/projects/048f09e433c343ba87faa257b37184c6` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:00.059591 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9838c6dc-e6aa-4f6a-ba3f-cfbaec50351e admin admin] RBAC: Authorizing `identity:delete_project(project_id=048f09e433c343ba87faa257b37184c6)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:00.061740 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9838c6dc-e6aa-4f6a-ba3f-cfbaec50351e admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:00.141893 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:00.203615 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 857/1706] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:53:59 2022] POST /identity/v3/users => generated 291 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:00.211582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ddc8c-7621-4ab2-8a8d-931372d31b71 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:00.236012 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-9838c6dc-e6aa-4f6a-ba3f-cfbaec50351e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '048f09e433c343ba87faa257b37184c6', 'request_id': 'req-9838c6dc-e6aa-4f6a-ba3f-cfbaec50351e'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:00.257045 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ddc8c-7621-4ab2-8a8d-931372d31b71 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:00.361881 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-9838c6dc-e6aa-4f6a-ba3f-cfbaec50351e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '048f09e433c343ba87faa257b37184c6', 'request_id': 'req-9838c6dc-e6aa-4f6a-ba3f-cfbaec50351e'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:00.393039 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-9838c6dc-e6aa-4f6a-ba3f-cfbaec50351e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '048f09e433c343ba87faa257b37184c6', 'request_id': 'req-9838c6dc-e6aa-4f6a-ba3f-cfbaec50351e'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:00.409023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a0c3b-a902-45cb-9695-aa663e6941a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:00.409935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a0c3b-a902-45cb-9695-aa663e6941a0 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:00.410354 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a0c3b-a902-45cb-9695-aa663e6941a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:00.411013 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a0c3b-a902-45cb-9695-aa663e6941a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:00.414771 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a0c3b-a902-45cb-9695-aa663e6941a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:00.417768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a0c3b-a902-45cb-9695-aa663e6941a0 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:00.441260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 858/1707] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:00 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:00.445558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-9838c6dc-e6aa-4f6a-ba3f-cfbaec50351e admin admin] The token cache is being invalidate because project 048f09e433c343ba87faa257b37184c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:54:00.446476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-9838c6dc-e6aa-4f6a-ba3f-cfbaec50351e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:00.449168 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a0c3b-a902-45cb-9695-aa663e6941a0 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:00.450949 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-9838c6dc-e6aa-4f6a-ba3f-cfbaec50351e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:00.458992 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-2f9a0c3b-a902-45cb-9695-aa663e6941a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:00.459972 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 850/1708] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:53:59 2022] DELETE /identity/v3/projects/048f09e433c343ba87faa257b37184c6 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:54:00.464290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838c6dc-e6aa-4f6a-ba3f-cfbaec50351e admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:00.471003 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-9838c6dc-e6aa-4f6a-ba3f-cfbaec50351e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:00.497404 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a0c3b-a902-45cb-9695-aa663e6941a0 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:00.506141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc91277-fff9-4202-90a0-2e247a30cc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:00.507211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc91277-fff9-4202-90a0-2e247a30cc6e admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:00.507778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc91277-fff9-4202-90a0-2e247a30cc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:00.508258 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc91277-fff9-4202-90a0-2e247a30cc6e admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:00.510677 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc91277-fff9-4202-90a0-2e247a30cc6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:00.512805 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc91277-fff9-4202-90a0-2e247a30cc6e admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:00.519958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838c6dc-e6aa-4f6a-ba3f-cfbaec50351e admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:00.527149 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b8ea9b-8c32-43a0-8692-7b739cc5684a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:00.530858 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 859/1709] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:00 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:00.532285 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc91277-fff9-4202-90a0-2e247a30cc6e admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:00.545499 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8ea9b-8c32-43a0-8692-7b739cc5684a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:00.546363 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8ea9b-8c32-43a0-8692-7b739cc5684a admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:00.546817 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8ea9b-8c32-43a0-8692-7b739cc5684a admin admin] PATH_INFO: `/v3/users/70bf957116dd41c39b51b57cd5377cc6` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:00.549562 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b8ea9b-8c32-43a0-8692-7b739cc5684a admin admin] RBAC: Authorizing `identity:delete_user(user_id=70bf957116dd41c39b51b57cd5377cc6)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:00.550028 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc91277-fff9-4202-90a0-2e247a30cc6e admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:00.553177 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b8ea9b-8c32-43a0-8692-7b739cc5684a admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:00.559149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4e913-cdf1-4cdb-8025-e2cfb45f83bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:00.560189 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4e913-cdf1-4cdb-8025-e2cfb45f83bb admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:00.561019 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4e913-cdf1-4cdb-8025-e2cfb45f83bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:00.561638 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4e913-cdf1-4cdb-8025-e2cfb45f83bb admin admin] PATH_INFO: `/v3/projects/557a9c6f50d640f1af2e920b05715907/users/9635007b04174749ba05a6879ca87b8f/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:00.584577 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4e913-cdf1-4cdb-8025-e2cfb45f83bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=557a9c6f50d640f1af2e920b05715907, user_id=9635007b04174749ba05a6879ca87b8f, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:00.588613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4e913-cdf1-4cdb-8025-e2cfb45f83bb admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:00.700604 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 860/1710] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:00 2022] PUT /identity/v3/projects/557a9c6f50d640f1af2e920b05715907/users/9635007b04174749ba05a6879ca87b8f/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:00.712713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b12c4b-277f-4c5d-b569-ea7fc069a518 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:00.713287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b12c4b-277f-4c5d-b569-ea7fc069a518 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:00.713751 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b12c4b-277f-4c5d-b569-ea7fc069a518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:00.774200 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-c6b12c4b-277f-4c5d-b569-ea7fc069a518 None None] MFA Rules not processed for user `9635007b04174749ba05a6879ca87b8f`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:00.776359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-c6b12c4b-277f-4c5d-b569-ea7fc069a518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:00.940820 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 861/1711] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:00.946477 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b12c4b-277f-4c5d-b569-ea7fc069a518 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:01.009128 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f4b8ea9b-8c32-43a0-8692-7b739cc5684a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70bf957116dd41c39b51b57cd5377cc6'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:01.015956 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:01.036416 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f4b8ea9b-8c32-43a0-8692-7b739cc5684a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70bf957116dd41c39b51b57cd5377cc6'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:01.124187 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b12c4b-277f-4c5d-b569-ea7fc069a518 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:01.130117 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a6c51-3897-4f94-9723-af31053ea90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:01.133959 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a6c51-3897-4f94-9723-af31053ea90b admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:01.133959 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a6c51-3897-4f94-9723-af31053ea90b admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:01.133959 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a6c51-3897-4f94-9723-af31053ea90b admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:01.133959 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a6c51-3897-4f94-9723-af31053ea90b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:01.135572 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a6c51-3897-4f94-9723-af31053ea90b admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:01.181836 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f4b8ea9b-8c32-43a0-8692-7b739cc5684a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70bf957116dd41c39b51b57cd5377cc6'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:01.183417 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f4b8ea9b-8c32-43a0-8692-7b739cc5684a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70bf957116dd41c39b51b57cd5377cc6'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:01.186797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f4b8ea9b-8c32-43a0-8692-7b739cc5684a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70bf957116dd41c39b51b57cd5377cc6'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:01.225480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 851/1712] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:54:00 2022] DELETE /identity/v3/users/70bf957116dd41c39b51b57cd5377cc6 => generated 0 bytes in 762 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:01.226555 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b8ea9b-8c32-43a0-8692-7b739cc5684a admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:01.228575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-f4b8ea9b-8c32-43a0-8692-7b739cc5684a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:01.241119 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 862/1713] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:54:00 2022] POST /identity/v3/projects => generated 377 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:54:01.246363 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a6c51-3897-4f94-9723-af31053ea90b admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:01.252350 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-c65a6c51-3897-4f94-9723-af31053ea90b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:01.382619 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a6c51-3897-4f94-9723-af31053ea90b admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:01.396795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f950a9-82ed-483f-9dda-adc85b2f940b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:01.397499 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f950a9-82ed-483f-9dda-adc85b2f940b admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:01.397780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f950a9-82ed-483f-9dda-adc85b2f940b admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:01.398048 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f950a9-82ed-483f-9dda-adc85b2f940b admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:01.399223 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f950a9-82ed-483f-9dda-adc85b2f940b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:01.404685 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f950a9-82ed-483f-9dda-adc85b2f940b admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:01.419100 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b8ea9b-8c32-43a0-8692-7b739cc5684a admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:01.443790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5dacb-6211-494e-82ca-315f2950e6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:01.444753 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5dacb-6211-494e-82ca-315f2950e6f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:01.446502 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5dacb-6211-494e-82ca-315f2950e6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:01.446926 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5dacb-6211-494e-82ca-315f2950e6f2 admin admin] PATH_INFO: `/v3/projects/b32342e98f1e4d6d8279e7a7d019c0fa` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:01.449704 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f5dacb-6211-494e-82ca-315f2950e6f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b32342e98f1e4d6d8279e7a7d019c0fa)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:01.451497 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f5dacb-6211-494e-82ca-315f2950e6f2 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:01.513324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 863/1714] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:54:01 2022] POST /identity/v3/users => generated 295 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:01.514729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae55b58-0672-43da-b263-05bd2d0fd1b3 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:01.515840 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae55b58-0672-43da-b263-05bd2d0fd1b3 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:01.515840 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae55b58-0672-43da-b263-05bd2d0fd1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:01.606610 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-35f5dacb-6211-494e-82ca-315f2950e6f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b32342e98f1e4d6d8279e7a7d019c0fa', 'request_id': 'req-35f5dacb-6211-494e-82ca-315f2950e6f2'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:01.629432 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-1ae55b58-0672-43da-b263-05bd2d0fd1b3 None None] MFA Rules not processed for user `bbd9bbb083ab4f5e84e135ec5103cdae`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:01.631947 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-1ae55b58-0672-43da-b263-05bd2d0fd1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:01.682486 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-35f5dacb-6211-494e-82ca-315f2950e6f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b32342e98f1e4d6d8279e7a7d019c0fa', 'request_id': 'req-35f5dacb-6211-494e-82ca-315f2950e6f2'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:01.691386 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-35f5dacb-6211-494e-82ca-315f2950e6f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b32342e98f1e4d6d8279e7a7d019c0fa', 'request_id': 'req-35f5dacb-6211-494e-82ca-315f2950e6f2'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:01.713969 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-35f5dacb-6211-494e-82ca-315f2950e6f2 admin admin] The token cache is being invalidate because project b32342e98f1e4d6d8279e7a7d019c0fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:54:01.715593 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-35f5dacb-6211-494e-82ca-315f2950e6f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:01.716939 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-35f5dacb-6211-494e-82ca-315f2950e6f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:01.720763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 852/1715] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:54:01 2022] DELETE /identity/v3/projects/b32342e98f1e4d6d8279e7a7d019c0fa => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:01.734967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c66962-59ec-4753-8533-6c5e98ac733b None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:01.734967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c66962-59ec-4753-8533-6c5e98ac733b None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:01.734967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c66962-59ec-4753-8533-6c5e98ac733b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:01.750182 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-f0c66962-59ec-4753-8533-6c5e98ac733b None None] MFA Rules not processed for user `34240c306dc54f0c8758dbb4b4525376`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:01.752701 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-f0c66962-59ec-4753-8533-6c5e98ac733b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:01.797924 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 864/1716] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Aug 26 00:54:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:01.800501 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae55b58-0672-43da-b263-05bd2d0fd1b3 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:01.804956 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-1ae55b58-0672-43da-b263-05bd2d0fd1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:01.838734 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 853/1717] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:01.840601 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a7bfe-1082-4c48-a58d-3728406c27c9 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:01.841148 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a7bfe-1082-4c48-a58d-3728406c27c9 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:01.841627 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a7bfe-1082-4c48-a58d-3728406c27c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:01.877122 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-276a7bfe-1082-4c48-a58d-3728406c27c9 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:01.878840 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-276a7bfe-1082-4c48-a58d-3728406c27c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:02.001721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 854/1718] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:02.011019 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547e33c-4847-4cf3-94db-f330ca99d9f9 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:02.011470 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547e33c-4847-4cf3-94db-f330ca99d9f9 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:02.011974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547e33c-4847-4cf3-94db-f330ca99d9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:02.077355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae55b58-0672-43da-b263-05bd2d0fd1b3 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:02.080282 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-a547e33c-4847-4cf3-94db-f330ca99d9f9 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:02.083612 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-a547e33c-4847-4cf3-94db-f330ca99d9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:02.098699 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bc54b-787c-4aa5-a78c-58e20fae14e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:02.100442 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bc54b-787c-4aa5-a78c-58e20fae14e0 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:02.100762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bc54b-787c-4aa5-a78c-58e20fae14e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:02.101045 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bc54b-787c-4aa5-a78c-58e20fae14e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:02.108913 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3bc54b-787c-4aa5-a78c-58e20fae14e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:02.114159 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 855/1719] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:02.115620 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3bc54b-787c-4aa5-a78c-58e20fae14e0 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:02.116502 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a547e33c-4847-4cf3-94db-f330ca99d9f9 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:02.141806 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a547e33c-4847-4cf3-94db-f330ca99d9f9 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:02.167504 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501660fb-7f4a-4f42-83b6-ec01ac82e0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:02.170905 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 865/1720] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:02.171122 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501660fb-7f4a-4f42-83b6-ec01ac82e0a5 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:02.171122 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501660fb-7f4a-4f42-83b6-ec01ac82e0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:02.171122 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501660fb-7f4a-4f42-83b6-ec01ac82e0a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:02.173598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bc54b-787c-4aa5-a78c-58e20fae14e0 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:02.174027 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501660fb-7f4a-4f42-83b6-ec01ac82e0a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:54:02.175749 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501660fb-7f4a-4f42-83b6-ec01ac82e0a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:02.180914 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501660fb-7f4a-4f42-83b6-ec01ac82e0a5 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:02.214102 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 856/1721] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:54:02.215296 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501660fb-7f4a-4f42-83b6-ec01ac82e0a5 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:02.215402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bc54b-787c-4aa5-a78c-58e20fae14e0 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:02.231960 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501660fb-7f4a-4f42-83b6-ec01ac82e0a5 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:02.237887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ec665-1ddb-4c48-9057-70cecf044288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:02.238888 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ec665-1ddb-4c48-9057-70cecf044288 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:02.239180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ec665-1ddb-4c48-9057-70cecf044288 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:02.239434 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb1c78-cc82-445b-a3af-38fcd3f227df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:02.239623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ec665-1ddb-4c48-9057-70cecf044288 admin admin] PATH_INFO: `/v3/projects/c17cc30e568e450a89d24188a861a500/users/3636d5458aee4fd1af1ab993afa45d6d/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:02.240972 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb1c78-cc82-445b-a3af-38fcd3f227df admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:02.241641 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb1c78-cc82-445b-a3af-38fcd3f227df admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:02.241975 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb1c78-cc82-445b-a3af-38fcd3f227df admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:02.242761 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb1c78-cc82-445b-a3af-38fcd3f227df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:02.245108 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb1c78-cc82-445b-a3af-38fcd3f227df admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:02.272518 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:02.299002 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ec665-1ddb-4c48-9057-70cecf044288 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c17cc30e568e450a89d24188a861a500, user_id=3636d5458aee4fd1af1ab993afa45d6d, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:02.301770 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ec665-1ddb-4c48-9057-70cecf044288 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:02.413246 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 857/1722] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:54:02 2022] POST /identity/v3/projects => generated 365 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:02.420513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb1c78-cc82-445b-a3af-38fcd3f227df admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:02.454406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 866/1723] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:02 2022] PUT /identity/v3/projects/c17cc30e568e450a89d24188a861a500/users/3636d5458aee4fd1af1ab993afa45d6d/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:02.457082 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ec665-1ddb-4c48-9057-70cecf044288 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:02.461028 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a52ec665-1ddb-4c48-9057-70cecf044288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:02.587063 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ec665-1ddb-4c48-9057-70cecf044288 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:02.604343 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb1c78-cc82-445b-a3af-38fcd3f227df admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:02.610182 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ab6eb-0373-44d1-a1e0-298783d9fb53 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f700502671446c797a566f07398a9fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:02.612327 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ab6eb-0373-44d1-a1e0-298783d9fb53 service swift] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:02.616556 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ab6eb-0373-44d1-a1e0-298783d9fb53 service swift] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:02.617784 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ab6eb-0373-44d1-a1e0-298783d9fb53 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:02.746534 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7118efb6-a070-4487-a1cf-c1954c6332d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:02.748428 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118efb6-a070-4487-a1cf-c1954c6332d9 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:02.748946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118efb6-a070-4487-a1cf-c1954c6332d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:02.749464 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118efb6-a070-4487-a1cf-c1954c6332d9 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:02.750619 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7118efb6-a070-4487-a1cf-c1954c6332d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:02.758539 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7118efb6-a070-4487-a1cf-c1954c6332d9 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:02.773039 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ab6eb-0373-44d1-a1e0-298783d9fb53 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:02.775183 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ab6eb-0373-44d1-a1e0-298783d9fb53 service swift] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:02.796444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 867/1724] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Aug 26 00:54:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 00:54:02.797209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ab6eb-0373-44d1-a1e0-298783d9fb53 service swift] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:02.809525 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ab6eb-0373-44d1-a1e0-298783d9fb53 service swift] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:02.816325 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5654e5-a2a8-449e-bc86-82b66e845e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:02.818022 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5654e5-a2a8-449e-bc86-82b66e845e1d admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:02.818022 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5654e5-a2a8-449e-bc86-82b66e845e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:02.818686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5654e5-a2a8-449e-bc86-82b66e845e1d admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:02.819103 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5654e5-a2a8-449e-bc86-82b66e845e1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:02.821527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5654e5-a2a8-449e-bc86-82b66e845e1d admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:02.835238 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 868/1725] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:02.836575 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea758b1e-672e-4f94-a2a8-552b12badb1f None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:02.836916 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea758b1e-672e-4f94-a2a8-552b12badb1f None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:02.837280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea758b1e-672e-4f94-a2a8-552b12badb1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:02.861924 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-ea758b1e-672e-4f94-a2a8-552b12badb1f None None] MFA Rules not processed for user `0175a2b440704d51aa14607a2e20e36a`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:02.863350 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-ea758b1e-672e-4f94-a2a8-552b12badb1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:02.969793 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 858/1726] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:54:02 2022] POST /identity/v3/users => generated 283 bytes in 550 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:02.979332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7118efb6-a070-4487-a1cf-c1954c6332d9 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:02.999590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 869/1727] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:03.001724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea758b1e-672e-4f94-a2a8-552b12badb1f None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:03.049565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7118efb6-a070-4487-a1cf-c1954c6332d9 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:03.086565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b6606-d022-43f9-a125-23f900e51cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:03.091583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b6606-d022-43f9-a125-23f900e51cb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:03.094673 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b6606-d022-43f9-a125-23f900e51cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:03.095928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b6606-d022-43f9-a125-23f900e51cb6 admin admin] PATH_INFO: `/v3/projects/c17cc30e568e450a89d24188a861a500/users/3636d5458aee4fd1af1ab993afa45d6d/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:03.097787 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea758b1e-672e-4f94-a2a8-552b12badb1f None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:03.115639 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b6606-d022-43f9-a125-23f900e51cb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c17cc30e568e450a89d24188a861a500, user_id=3636d5458aee4fd1af1ab993afa45d6d, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:03.125618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7dc306-faaa-4f14-801e-5f88bf5dd8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:03.129240 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7dc306-faaa-4f14-801e-5f88bf5dd8ee admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:03.132075 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7dc306-faaa-4f14-801e-5f88bf5dd8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:03.132361 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7dc306-faaa-4f14-801e-5f88bf5dd8ee admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:03.133286 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7dc306-faaa-4f14-801e-5f88bf5dd8ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:03.135263 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b6606-d022-43f9-a125-23f900e51cb6 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:03.137816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7dc306-faaa-4f14-801e-5f88bf5dd8ee admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:03.175482 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 870/1728] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:03.176887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7dc306-faaa-4f14-801e-5f88bf5dd8ee admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:03.186205 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-0a7dc306-faaa-4f14-801e-5f88bf5dd8ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:03.196329 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 859/1729] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:02 2022] PUT /identity/v3/projects/c17cc30e568e450a89d24188a861a500/users/3636d5458aee4fd1af1ab993afa45d6d/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:03.206257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b6606-d022-43f9-a125-23f900e51cb6 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:03.217349 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7dc306-faaa-4f14-801e-5f88bf5dd8ee admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:03.328911 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:03.446880 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c4272-8558-4194-b25b-5dd21d8a3491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:03.447525 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c4272-8558-4194-b25b-5dd21d8a3491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:03.447820 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c4272-8558-4194-b25b-5dd21d8a3491 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:03.448081 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c4272-8558-4194-b25b-5dd21d8a3491 admin admin] PATH_INFO: `/v3/projects/ca1a99f0dbec4b7bab2d2078c36cb080` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:03.449855 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c4272-8558-4194-b25b-5dd21d8a3491 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca1a99f0dbec4b7bab2d2078c36cb080)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:03.452004 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c4272-8558-4194-b25b-5dd21d8a3491 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:03.472031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b6606-d022-43f9-a125-23f900e51cb6 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:03.499295 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3181b-5e88-41ee-8661-fc169409aa6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:03.500234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3181b-5e88-41ee-8661-fc169409aa6c admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:03.507341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3181b-5e88-41ee-8661-fc169409aa6c admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:03.507341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3181b-5e88-41ee-8661-fc169409aa6c admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:03.507341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3181b-5e88-41ee-8661-fc169409aa6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:03.507341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3181b-5e88-41ee-8661-fc169409aa6c admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:03.522654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 860/1730] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:03.532777 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8076ef1-93dc-44c5-b8b4-da4a50de03fd None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:03.533684 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8076ef1-93dc-44c5-b8b4-da4a50de03fd None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:03.534171 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8076ef1-93dc-44c5-b8b4-da4a50de03fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:03.631529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-a8076ef1-93dc-44c5-b8b4-da4a50de03fd None None] MFA Rules not processed for user `3636d5458aee4fd1af1ab993afa45d6d`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:03.633294 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-a8076ef1-93dc-44c5-b8b4-da4a50de03fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:03.649472 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-1d0c4272-8558-4194-b25b-5dd21d8a3491 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca1a99f0dbec4b7bab2d2078c36cb080', 'request_id': 'req-1d0c4272-8558-4194-b25b-5dd21d8a3491'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:03.809526 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-1d0c4272-8558-4194-b25b-5dd21d8a3491 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca1a99f0dbec4b7bab2d2078c36cb080', 'request_id': 'req-1d0c4272-8558-4194-b25b-5dd21d8a3491'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:03.815836 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-1d0c4272-8558-4194-b25b-5dd21d8a3491 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca1a99f0dbec4b7bab2d2078c36cb080', 'request_id': 'req-1d0c4272-8558-4194-b25b-5dd21d8a3491'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:03.880047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-1d0c4272-8558-4194-b25b-5dd21d8a3491 admin admin] The token cache is being invalidate because project ca1a99f0dbec4b7bab2d2078c36cb080 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:54:03.880493 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-1d0c4272-8558-4194-b25b-5dd21d8a3491 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:03.885666 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-1d0c4272-8558-4194-b25b-5dd21d8a3491 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:03.892390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 871/1731] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:54:03 2022] DELETE /identity/v3/projects/ca1a99f0dbec4b7bab2d2078c36cb080 => generated 0 bytes in 716 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:03.922129 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 861/1732] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:03.922211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c4272-8558-4194-b25b-5dd21d8a3491 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:03.922282 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f09c7e-5ff8-4172-aff6-eee3ab253577 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:03.922282 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f09c7e-5ff8-4172-aff6-eee3ab253577 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:03.922282 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f09c7e-5ff8-4172-aff6-eee3ab253577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:03.922458 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-1d0c4272-8558-4194-b25b-5dd21d8a3491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:03.958816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-94f09c7e-5ff8-4172-aff6-eee3ab253577 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:03.963083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-94f09c7e-5ff8-4172-aff6-eee3ab253577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:04.125659 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c4272-8558-4194-b25b-5dd21d8a3491 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:04.148862 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae12122-9744-44b9-b624-10521b84f13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:04.151473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae12122-9744-44b9-b624-10521b84f13d admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:04.153034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae12122-9744-44b9-b624-10521b84f13d admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:04.155479 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae12122-9744-44b9-b624-10521b84f13d admin admin] PATH_INFO: `/v3/projects/b17ed5a9456e48a29e8851fc66da69f5/users/721215eaa8164f0c8a2049dd5581d89c/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:04.183008 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 862/1733] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:04.183008 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f09c7e-5ff8-4172-aff6-eee3ab253577 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:04.183998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-94f09c7e-5ff8-4172-aff6-eee3ab253577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:04.202636 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae12122-9744-44b9-b624-10521b84f13d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b17ed5a9456e48a29e8851fc66da69f5, user_id=721215eaa8164f0c8a2049dd5581d89c, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:04.227188 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae12122-9744-44b9-b624-10521b84f13d admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:04.233097 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:04.354076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f09c7e-5ff8-4172-aff6-eee3ab253577 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:04.365286 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 872/1734] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:03 2022] PUT /identity/v3/projects/b17ed5a9456e48a29e8851fc66da69f5/users/721215eaa8164f0c8a2049dd5581d89c/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:04.368711 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74faa342-82f9-4d3e-a7cd-3524b0a2943f None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:04.369799 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74faa342-82f9-4d3e-a7cd-3524b0a2943f None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:04.371315 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74faa342-82f9-4d3e-a7cd-3524b0a2943f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:04.476055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-74faa342-82f9-4d3e-a7cd-3524b0a2943f None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:04.666204 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-74faa342-82f9-4d3e-a7cd-3524b0a2943f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:04.667736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167715a-d404-4d1f-8181-232b238aaed9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:04.667736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167715a-d404-4d1f-8181-232b238aaed9 service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:04.667736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167715a-d404-4d1f-8181-232b238aaed9 service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:04.667736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167715a-d404-4d1f-8181-232b238aaed9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:04.668780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-6167715a-d404-4d1f-8181-232b238aaed9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:04.669243 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 873/1735] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:04.669243 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c4ee0-8cc0-4403-a006-18aad9d9c534 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:04.669243 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c4ee0-8cc0-4403-a006-18aad9d9c534 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:04.669243 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c4ee0-8cc0-4403-a006-18aad9d9c534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:04.669243 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-ca2c4ee0-8cc0-4403-a006-18aad9d9c534 None None] MFA Rules not processed for user `721215eaa8164f0c8a2049dd5581d89c`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:04.669243 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-ca2c4ee0-8cc0-4403-a006-18aad9d9c534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:04.726509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167715a-d404-4d1f-8181-232b238aaed9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:04.726509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167715a-d404-4d1f-8181-232b238aaed9 service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:04.741206 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 863/1736] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:54:04.750507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167715a-d404-4d1f-8181-232b238aaed9 service cinder] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:04.794869 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167715a-d404-4d1f-8181-232b238aaed9 service cinder] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:04.814419 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe4d57-7807-4d9c-9caa-c2182eb4607a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:04.815060 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe4d57-7807-4d9c-9caa-c2182eb4607a admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:04.815343 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe4d57-7807-4d9c-9caa-c2182eb4607a admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:04.815616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe4d57-7807-4d9c-9caa-c2182eb4607a admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:04.816291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe4d57-7807-4d9c-9caa-c2182eb4607a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:54:04.816701 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe4d57-7807-4d9c-9caa-c2182eb4607a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:04.818318 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe4d57-7807-4d9c-9caa-c2182eb4607a admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:04.851868 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 864/1737] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:54:04.853072 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe4d57-7807-4d9c-9caa-c2182eb4607a admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:04.875677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe4d57-7807-4d9c-9caa-c2182eb4607a admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:04.886158 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdeb575-fc53-42da-b871-044b8951f3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:04.887373 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdeb575-fc53-42da-b871-044b8951f3a5 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:04.887373 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdeb575-fc53-42da-b871-044b8951f3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:04.887373 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdeb575-fc53-42da-b871-044b8951f3a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:04.887945 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdeb575-fc53-42da-b871-044b8951f3a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:04.892435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdeb575-fc53-42da-b871-044b8951f3a5 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:04.921300 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 874/1738] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:04.927485 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c4ee0-8cc0-4403-a006-18aad9d9c534 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:04.991143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c4ee0-8cc0-4403-a006-18aad9d9c534 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:05.032528 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dafefd-85f7-4462-a072-f89c6931c3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:05.032867 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dafefd-85f7-4462-a072-f89c6931c3a9 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:05.033201 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dafefd-85f7-4462-a072-f89c6931c3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:05.033481 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dafefd-85f7-4462-a072-f89c6931c3a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:05.034348 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dafefd-85f7-4462-a072-f89c6931c3a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:05.037814 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dafefd-85f7-4462-a072-f89c6931c3a9 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:05.046057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 865/1739] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:54:04 2022] POST /identity/v3/projects => generated 365 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:05.053397 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdeb575-fc53-42da-b871-044b8951f3a5 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:05.223094 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 875/1740] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:54:04 2022] POST /identity/v3/projects => generated 365 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:05.228513 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dafefd-85f7-4462-a072-f89c6931c3a9 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:05.274156 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdeb575-fc53-42da-b871-044b8951f3a5 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:05.492793 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc095c-805f-4243-aaf3-0bf484b7049d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:05.500650 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc095c-805f-4243-aaf3-0bf484b7049d admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:05.500650 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc095c-805f-4243-aaf3-0bf484b7049d admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:05.500650 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc095c-805f-4243-aaf3-0bf484b7049d admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:05.500650 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cc095c-805f-4243-aaf3-0bf484b7049d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:05.500650 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cc095c-805f-4243-aaf3-0bf484b7049d admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:05.514960 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dafefd-85f7-4462-a072-f89c6931c3a9 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:05.543193 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003aa5b-159e-4bf1-b443-3d330c06fc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:05.544138 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003aa5b-159e-4bf1-b443-3d330c06fc36 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:05.544588 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003aa5b-159e-4bf1-b443-3d330c06fc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:05.545009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003aa5b-159e-4bf1-b443-3d330c06fc36 admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:05.545790 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003aa5b-159e-4bf1-b443-3d330c06fc36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:05.547905 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003aa5b-159e-4bf1-b443-3d330c06fc36 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:05.670688 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 866/1741] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:54:05 2022] POST /identity/v3/users => generated 283 bytes in 617 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:05.672157 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd24fd3-dd82-4248-bff4-8b58bff549a7 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:05.672452 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd24fd3-dd82-4248-bff4-8b58bff549a7 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:05.675996 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd24fd3-dd82-4248-bff4-8b58bff549a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:05.685244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 876/1742] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:54:05 2022] POST /identity/v3/users => generated 289 bytes in 457 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:05.686041 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003aa5b-159e-4bf1-b443-3d330c06fc36 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:05.718351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003aa5b-159e-4bf1-b443-3d330c06fc36 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:05.739067 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c24b950-5188-4420-a3a0-6448c9bc195f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:05.739824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24b950-5188-4420-a3a0-6448c9bc195f admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:05.740106 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24b950-5188-4420-a3a0-6448c9bc195f admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:05.740380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24b950-5188-4420-a3a0-6448c9bc195f admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:05.745663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c24b950-5188-4420-a3a0-6448c9bc195f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:05.758877 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c24b950-5188-4420-a3a0-6448c9bc195f admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:05.772459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:05.779261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 877/1743] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:05 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:05.780491 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c24b950-5188-4420-a3a0-6448c9bc195f admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:05.783338 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-afd24fd3-dd82-4248-bff4-8b58bff549a7 None None] MFA Rules not processed for user `4698a49ed4a04f58a488359a2742e5da`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:05.784923 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-afd24fd3-dd82-4248-bff4-8b58bff549a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:05.811763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c24b950-5188-4420-a3a0-6448c9bc195f admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:05.820986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2b027-f350-489b-bbdb-1b0ad0492620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:05.824195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2b027-f350-489b-bbdb-1b0ad0492620 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:05.824195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2b027-f350-489b-bbdb-1b0ad0492620 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:05.824195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2b027-f350-489b-bbdb-1b0ad0492620 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:05.824195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2b027-f350-489b-bbdb-1b0ad0492620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:05.833226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2b027-f350-489b-bbdb-1b0ad0492620 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:05.857503 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:05.883635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 878/1744] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:05 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:05.886994 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2b027-f350-489b-bbdb-1b0ad0492620 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:05.919040 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2b027-f350-489b-bbdb-1b0ad0492620 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:05.931842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0a9bf-9af8-4299-9db6-63399f285d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:05.934091 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0a9bf-9af8-4299-9db6-63399f285d52 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:05.935340 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0a9bf-9af8-4299-9db6-63399f285d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:05.936407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0a9bf-9af8-4299-9db6-63399f285d52 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:05.938990 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c0a9bf-9af8-4299-9db6-63399f285d52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:05.945250 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c0a9bf-9af8-4299-9db6-63399f285d52 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:05.967324 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 867/1745] 10.222.0.91 () {58 vars in 1015 bytes} [Fri Aug 26 00:54:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:05.973849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd24fd3-dd82-4248-bff4-8b58bff549a7 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:05.982057 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 879/1746] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:05 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:05.983169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0a9bf-9af8-4299-9db6-63399f285d52 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:05.987102 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd24fd3-dd82-4248-bff4-8b58bff549a7 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:05.997041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965d39f-e179-4282-93a6-12a2c101a1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:05.998751 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965d39f-e179-4282-93a6-12a2c101a1f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:05.999224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965d39f-e179-4282-93a6-12a2c101a1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:05.999856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965d39f-e179-4282-93a6-12a2c101a1f8 admin admin] PATH_INFO: `/v3/projects/7ec5fa515a094c6189157a2dc006863c/users/88157f1e0abb4ff393a14efaf0b2e77c/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:06.026824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9965d39f-e179-4282-93a6-12a2c101a1f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ec5fa515a094c6189157a2dc006863c, user_id=88157f1e0abb4ff393a14efaf0b2e77c, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:06.031271 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9965d39f-e179-4282-93a6-12a2c101a1f8 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:06.032494 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0a9bf-9af8-4299-9db6-63399f285d52 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:06.063215 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeba57c-4ffd-47bb-b72e-4f6152fc215e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:06.063917 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeba57c-4ffd-47bb-b72e-4f6152fc215e admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:06.064212 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeba57c-4ffd-47bb-b72e-4f6152fc215e admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:06.064516 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeba57c-4ffd-47bb-b72e-4f6152fc215e admin admin] PATH_INFO: `/v3/projects/2f306ac368f04a8eaa4b00e1c5d1ae23/users/06cc2a4e266b450bae22fc66ab711be5/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:06.121244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeba57c-4ffd-47bb-b72e-4f6152fc215e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f306ac368f04a8eaa4b00e1c5d1ae23, user_id=06cc2a4e266b450bae22fc66ab711be5, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:06.124326 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeba57c-4ffd-47bb-b72e-4f6152fc215e admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:06.144308 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 868/1747] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:05 2022] PUT /identity/v3/projects/7ec5fa515a094c6189157a2dc006863c/users/88157f1e0abb4ff393a14efaf0b2e77c/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:06.144308 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965d39f-e179-4282-93a6-12a2c101a1f8 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:06.147949 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-9965d39f-e179-4282-93a6-12a2c101a1f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:06.236918 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 880/1748] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:05 2022] PUT /identity/v3/projects/2f306ac368f04a8eaa4b00e1c5d1ae23/users/06cc2a4e266b450bae22fc66ab711be5/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:06.237983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeba57c-4ffd-47bb-b72e-4f6152fc215e admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:06.351954 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeba57c-4ffd-47bb-b72e-4f6152fc215e admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:06.358117 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb35c6-ab21-4b4a-bc8f-fe4a3174f544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:06.358934 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb35c6-ab21-4b4a-bc8f-fe4a3174f544 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:06.359646 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb35c6-ab21-4b4a-bc8f-fe4a3174f544 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:06.360122 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb35c6-ab21-4b4a-bc8f-fe4a3174f544 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:06.361021 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb35c6-ab21-4b4a-bc8f-fe4a3174f544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:06.363739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb35c6-ab21-4b4a-bc8f-fe4a3174f544 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:06.372555 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965d39f-e179-4282-93a6-12a2c101a1f8 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:06.380946 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 881/1749] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:06.388044 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb35c6-ab21-4b4a-bc8f-fe4a3174f544 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:06.562721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5cf05b-2b47-442a-8861-2e49cf8d3313 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:06.563741 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cf05b-2b47-442a-8861-2e49cf8d3313 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:06.564203 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cf05b-2b47-442a-8861-2e49cf8d3313 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:06.564705 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cf05b-2b47-442a-8861-2e49cf8d3313 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:06.574247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb35c6-ab21-4b4a-bc8f-fe4a3174f544 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:06.581118 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5cf05b-2b47-442a-8861-2e49cf8d3313 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:06.583828 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5cf05b-2b47-442a-8861-2e49cf8d3313 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:06.592064 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2c32e-ec9f-403c-8f0b-57257b086b7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:06.592064 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2c32e-ec9f-403c-8f0b-57257b086b7b service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:06.593098 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2c32e-ec9f-403c-8f0b-57257b086b7b service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:06.593433 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2c32e-ec9f-403c-8f0b-57257b086b7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:06.633014 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 869/1750] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:54:06.633441 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d2c32e-ec9f-403c-8f0b-57257b086b7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:06.636156 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d2c32e-ec9f-403c-8f0b-57257b086b7b service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:06.636880 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59be97-99f6-4495-8a40-4a17a298a2e9 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:06.637277 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59be97-99f6-4495-8a40-4a17a298a2e9 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:06.637652 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59be97-99f6-4495-8a40-4a17a298a2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:06.649570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 882/1751] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:54:06.650901 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2c32e-ec9f-403c-8f0b-57257b086b7b service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:06.652837 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-57d2c32e-ec9f-403c-8f0b-57257b086b7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:06.756514 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-fa59be97-99f6-4495-8a40-4a17a298a2e9 None None] MFA Rules not processed for user `06cc2a4e266b450bae22fc66ab711be5`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:06.756514 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-fa59be97-99f6-4495-8a40-4a17a298a2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:06.757212 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2c32e-ec9f-403c-8f0b-57257b086b7b service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:06.770456 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9a934-39b0-407f-8dcc-faf7c5f6c89c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:06.772153 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9a934-39b0-407f-8dcc-faf7c5f6c89c service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:06.774256 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9a934-39b0-407f-8dcc-faf7c5f6c89c service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:06.774903 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9a934-39b0-407f-8dcc-faf7c5f6c89c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:06.792405 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe9a934-39b0-407f-8dcc-faf7c5f6c89c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:06.793370 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe9a934-39b0-407f-8dcc-faf7c5f6c89c service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:06.825760 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 883/1752] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:54:06.827017 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9a934-39b0-407f-8dcc-faf7c5f6c89c service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:06.870068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9a934-39b0-407f-8dcc-faf7c5f6c89c service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:06.879035 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bde92-a1ba-4678-b3bb-1f0d47dbabf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:06.882649 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bde92-a1ba-4678-b3bb-1f0d47dbabf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:06.884298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bde92-a1ba-4678-b3bb-1f0d47dbabf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:06.885067 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bde92-a1ba-4678-b3bb-1f0d47dbabf0 admin admin] PATH_INFO: `/v3/projects/7ec5fa515a094c6189157a2dc006863c/users/88157f1e0abb4ff393a14efaf0b2e77c/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:06.888790 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37bde92-a1ba-4678-b3bb-1f0d47dbabf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ec5fa515a094c6189157a2dc006863c, user_id=88157f1e0abb4ff393a14efaf0b2e77c, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:06.893073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37bde92-a1ba-4678-b3bb-1f0d47dbabf0 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:06.973079 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 870/1753] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:06.975574 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59be97-99f6-4495-8a40-4a17a298a2e9 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:06.986907 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 884/1754] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:06 2022] PUT /identity/v3/projects/7ec5fa515a094c6189157a2dc006863c/users/88157f1e0abb4ff393a14efaf0b2e77c/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:06.998883 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bde92-a1ba-4678-b3bb-1f0d47dbabf0 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:07.206022 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bde92-a1ba-4678-b3bb-1f0d47dbabf0 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:07.221255 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c67204-d6df-4402-aaf4-cda086817a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:07.222141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c67204-d6df-4402-aaf4-cda086817a45 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:07.222563 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c67204-d6df-4402-aaf4-cda086817a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:07.222991 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c67204-d6df-4402-aaf4-cda086817a45 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:07.223850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c67204-d6df-4402-aaf4-cda086817a45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:07.225967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59be97-99f6-4495-8a40-4a17a298a2e9 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:07.231885 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c67204-d6df-4402-aaf4-cda086817a45 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:07.374158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 885/1755] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:54:06 2022] POST /identity/v3/projects => generated 363 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:07.375859 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e28eaf-1cad-4f19-b7eb-f6093a938e8c None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:07.376356 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e28eaf-1cad-4f19-b7eb-f6093a938e8c None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:07.376889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e28eaf-1cad-4f19-b7eb-f6093a938e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:07.456570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-b1e28eaf-1cad-4f19-b7eb-f6093a938e8c None None] MFA Rules not processed for user `88157f1e0abb4ff393a14efaf0b2e77c`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:07.456570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-b1e28eaf-1cad-4f19-b7eb-f6093a938e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:07.461100 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e761d381-9157-4aa3-a593-fbb7d7ac0197 tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3636d5458aee4fd1af1ab993afa45d6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c17cc30e568e450a89d24188a861a500', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:07.461816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761d381-9157-4aa3-a593-fbb7d7ac0197 tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:07.462109 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761d381-9157-4aa3-a593-fbb7d7ac0197 tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:07.462390 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761d381-9157-4aa3-a593-fbb7d7ac0197 tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:07.463071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e761d381-9157-4aa3-a593-fbb7d7ac0197 tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:54:07.463417 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e761d381-9157-4aa3-a593-fbb7d7ac0197 tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:07.465098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e761d381-9157-4aa3-a593-fbb7d7ac0197 tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:07.561896 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:07.567688 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 871/1756] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 594 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:54:07.568298 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e761d381-9157-4aa3-a593-fbb7d7ac0197 tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:07.610358 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e761d381-9157-4aa3-a593-fbb7d7ac0197 tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:07.657783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 886/1757] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:07 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:07.660325 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e28eaf-1cad-4f19-b7eb-f6093a938e8c None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:07.885950 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e28eaf-1cad-4f19-b7eb-f6093a938e8c None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:07.891261 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e7a1d-1a26-42cf-8755-b92a8082ec2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:07.893425 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e7a1d-1a26-42cf-8755-b92a8082ec2c admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:07.893425 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e7a1d-1a26-42cf-8755-b92a8082ec2c admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:07.893425 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e7a1d-1a26-42cf-8755-b92a8082ec2c admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:07.894424 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e7a1d-1a26-42cf-8755-b92a8082ec2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:07.898404 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e7a1d-1a26-42cf-8755-b92a8082ec2c admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:07.901536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445ef6d-e7f2-4252-b797-b88720b8c403 tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3636d5458aee4fd1af1ab993afa45d6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c17cc30e568e450a89d24188a861a500', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:07.902369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445ef6d-e7f2-4252-b797-b88720b8c403 tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:07.902726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445ef6d-e7f2-4252-b797-b88720b8c403 tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:07.904952 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445ef6d-e7f2-4252-b797-b88720b8c403 tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:07.905864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445ef6d-e7f2-4252-b797-b88720b8c403 tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:07.912082 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445ef6d-e7f2-4252-b797-b88720b8c403 tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:07.975808 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 872/1758] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:54:07 2022] POST /identity/v3/users => generated 288 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:07.979690 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e7a1d-1a26-42cf-8755-b92a8082ec2c admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:08.040031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 887/1759] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:54:07 2022] POST /identity/v3/projects => generated 354 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:08.041025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445ef6d-e7f2-4252-b797-b88720b8c403 tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:08.078239 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:08.151525 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:08.187787 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e7a1d-1a26-42cf-8755-b92a8082ec2c admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:08.266180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445ef6d-e7f2-4252-b797-b88720b8c403 tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:08.288546 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b91ff1-78f4-475d-a0d4-83636fbb6ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:08.289492 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b91ff1-78f4-475d-a0d4-83636fbb6ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:08.290130 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b91ff1-78f4-475d-a0d4-83636fbb6ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:08.290980 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b91ff1-78f4-475d-a0d4-83636fbb6ad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:08.292122 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b91ff1-78f4-475d-a0d4-83636fbb6ad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:08.294031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b91ff1-78f4-475d-a0d4-83636fbb6ad7 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:08.323677 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 888/1760] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:08.324306 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b91ff1-78f4-475d-a0d4-83636fbb6ad7 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:08.349308 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15f283-5374-4f01-a7b0-f46c0455cbf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:08.350031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15f283-5374-4f01-a7b0-f46c0455cbf9 service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:08.353850 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15f283-5374-4f01-a7b0-f46c0455cbf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:08.353850 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15f283-5374-4f01-a7b0-f46c0455cbf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:08.477675 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b91ff1-78f4-475d-a0d4-83636fbb6ad7 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:08.484322 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a798303-7c8e-4763-99de-9d45cd60ed86 tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3636d5458aee4fd1af1ab993afa45d6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c17cc30e568e450a89d24188a861a500', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:08.484322 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a798303-7c8e-4763-99de-9d45cd60ed86 tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:08.484322 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a798303-7c8e-4763-99de-9d45cd60ed86 tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:08.484632 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a798303-7c8e-4763-99de-9d45cd60ed86 tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:08.491030 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a798303-7c8e-4763-99de-9d45cd60ed86 tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:54:08.492023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a798303-7c8e-4763-99de-9d45cd60ed86 tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:08.493786 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a798303-7c8e-4763-99de-9d45cd60ed86 tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:08.518320 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 889/1761] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:54:08.529158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a798303-7c8e-4763-99de-9d45cd60ed86 tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:08.553876 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a798303-7c8e-4763-99de-9d45cd60ed86 tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:08.569281 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa9d08-4cde-4c44-add5-ef96333471e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:08.581535 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa9d08-4cde-4c44-add5-ef96333471e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:08.581535 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa9d08-4cde-4c44-add5-ef96333471e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:08.581535 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa9d08-4cde-4c44-add5-ef96333471e9 admin admin] PATH_INFO: `/v3/projects/0fd76bc578364a74970d940693e7c2aa/users/3f3a3d16822d43b3814c0fd4baba8fb1/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:08.596049 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab15f283-5374-4f01-a7b0-f46c0455cbf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:08.599397 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab15f283-5374-4f01-a7b0-f46c0455cbf9 service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:08.616332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 873/1762] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:07 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:54:08.618409 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa9d08-4cde-4c44-add5-ef96333471e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fd76bc578364a74970d940693e7c2aa, user_id=3f3a3d16822d43b3814c0fd4baba8fb1, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:08.621689 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa9d08-4cde-4c44-add5-ef96333471e9 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:08.655229 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15f283-5374-4f01-a7b0-f46c0455cbf9 service cinder] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:08.708687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15f283-5374-4f01-a7b0-f46c0455cbf9 service cinder] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:08.722087 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794babcc-370a-4ac9-98d1-b7005411cffa tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3636d5458aee4fd1af1ab993afa45d6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c17cc30e568e450a89d24188a861a500', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:08.723103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794babcc-370a-4ac9-98d1-b7005411cffa tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:08.723630 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794babcc-370a-4ac9-98d1-b7005411cffa tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:08.724100 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794babcc-370a-4ac9-98d1-b7005411cffa tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] PATH_INFO: `/v3/projects/819a39d615324431b998bd2035ab7eab` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:08.726547 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794babcc-370a-4ac9-98d1-b7005411cffa tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] RBAC: Authorizing `identity:delete_project(project_id=819a39d615324431b998bd2035ab7eab)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:08.730144 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794babcc-370a-4ac9-98d1-b7005411cffa tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:08.742212 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 890/1763] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:08 2022] PUT /identity/v3/projects/0fd76bc578364a74970d940693e7c2aa/users/3f3a3d16822d43b3814c0fd4baba8fb1/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:08.749715 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa9d08-4cde-4c44-add5-ef96333471e9 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:08.979179 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-794babcc-370a-4ac9-98d1-b7005411cffa tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '819a39d615324431b998bd2035ab7eab', 'request_id': 'req-794babcc-370a-4ac9-98d1-b7005411cffa'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:09.001599 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa9d08-4cde-4c44-add5-ef96333471e9 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:09.044218 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-794babcc-370a-4ac9-98d1-b7005411cffa tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '819a39d615324431b998bd2035ab7eab', 'request_id': 'req-794babcc-370a-4ac9-98d1-b7005411cffa'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:09.052783 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-794babcc-370a-4ac9-98d1-b7005411cffa tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '819a39d615324431b998bd2035ab7eab', 'request_id': 'req-794babcc-370a-4ac9-98d1-b7005411cffa'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:09.140963 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-794babcc-370a-4ac9-98d1-b7005411cffa tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] The token cache is being invalidate because project 819a39d615324431b998bd2035ab7eab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:54:09.140963 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-794babcc-370a-4ac9-98d1-b7005411cffa tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:09.140963 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-794babcc-370a-4ac9-98d1-b7005411cffa tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:09.204201 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7098ed27-b61d-47b7-8ce9-32cf4f66762b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:09.204201 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098ed27-b61d-47b7-8ce9-32cf4f66762b admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:09.204201 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098ed27-b61d-47b7-8ce9-32cf4f66762b admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:09.204201 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098ed27-b61d-47b7-8ce9-32cf4f66762b admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:09.204500 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7098ed27-b61d-47b7-8ce9-32cf4f66762b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:09.204500 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7098ed27-b61d-47b7-8ce9-32cf4f66762b admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:09.204644 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 874/1764] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:54:08 2022] DELETE /identity/v3/projects/819a39d615324431b998bd2035ab7eab => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:09.204715 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 891/1765] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 446 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:09.215052 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794babcc-370a-4ac9-98d1-b7005411cffa tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:09.228222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-794babcc-370a-4ac9-98d1-b7005411cffa tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:09.284506 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794babcc-370a-4ac9-98d1-b7005411cffa tempest-VolumeQuotaClassesTest-59983042 tempest-VolumeQuotaClassesTest-59983042-project-admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:09.301324 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036ab51-98c4-43b8-ae18-f40672d6a979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:09.302766 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036ab51-98c4-43b8-ae18-f40672d6a979 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:09.303147 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036ab51-98c4-43b8-ae18-f40672d6a979 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:09.303361 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036ab51-98c4-43b8-ae18-f40672d6a979 admin admin] PATH_INFO: `/v3/projects/0fd76bc578364a74970d940693e7c2aa/users/3f3a3d16822d43b3814c0fd4baba8fb1/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:09.305682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036ab51-98c4-43b8-ae18-f40672d6a979 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fd76bc578364a74970d940693e7c2aa, user_id=3f3a3d16822d43b3814c0fd4baba8fb1, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:09.308575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036ab51-98c4-43b8-ae18-f40672d6a979 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:09.339292 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 875/1766] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:09 2022] PUT /identity/v3/projects/0fd76bc578364a74970d940693e7c2aa/users/3f3a3d16822d43b3814c0fd4baba8fb1/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:09.341279 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139b396-19db-4955-8a7e-22cbc975938d None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:09.341648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139b396-19db-4955-8a7e-22cbc975938d None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:09.341924 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139b396-19db-4955-8a7e-22cbc975938d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:09.456201 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-6139b396-19db-4955-8a7e-22cbc975938d None None] MFA Rules not processed for user `3f3a3d16822d43b3814c0fd4baba8fb1`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:09.457310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-6139b396-19db-4955-8a7e-22cbc975938d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:09.714555 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 892/1767] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:09 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:09.768843 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba713da6-ad05-44bc-9fa9-51e2226d6950 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:09.769138 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba713da6-ad05-44bc-9fa9-51e2226d6950 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:09.771787 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba713da6-ad05-44bc-9fa9-51e2226d6950 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:09.775773 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 876/1768] 10.222.0.59 () {54 vars in 921 bytes} [Fri Aug 26 00:54:09 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:54:09.818429 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086dbcc-32ec-489e-b389-1a0f2fa6deb6 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:09.819098 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086dbcc-32ec-489e-b389-1a0f2fa6deb6 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:09.819098 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086dbcc-32ec-489e-b389-1a0f2fa6deb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:09.869756 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-2086dbcc-32ec-489e-b389-1a0f2fa6deb6 None None] MFA Rules not processed for user `0602593263ca4b05a657d5b7697f932f`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:09.873873 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-2086dbcc-32ec-489e-b389-1a0f2fa6deb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:09.900247 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:10.125283 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba713da6-ad05-44bc-9fa9-51e2226d6950 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:10.127540 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 893/1769] 10.222.0.59 () {58 vars in 1018 bytes} [Fri Aug 26 00:54:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:10.128423 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-ba713da6-ad05-44bc-9fa9-51e2226d6950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:10.270419 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba713da6-ad05-44bc-9fa9-51e2226d6950 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:10.274854 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9941752e-1350-4e69-81d8-d5a2f0512f8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:10.276465 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941752e-1350-4e69-81d8-d5a2f0512f8b service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:10.277218 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941752e-1350-4e69-81d8-d5a2f0512f8b service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:10.278292 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941752e-1350-4e69-81d8-d5a2f0512f8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:10.323813 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9941752e-1350-4e69-81d8-d5a2f0512f8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:10.325618 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9941752e-1350-4e69-81d8-d5a2f0512f8b service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:10.376846 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 877/1770] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:10 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:54:10.485070 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:10.493583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086dbcc-32ec-489e-b389-1a0f2fa6deb6 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:10.495345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-2086dbcc-32ec-489e-b389-1a0f2fa6deb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:10.524518 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086dbcc-32ec-489e-b389-1a0f2fa6deb6 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:10.532521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b2b8f-0941-4674-abce-044a67b4f725 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:10.533388 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b2b8f-0941-4674-abce-044a67b4f725 service nova] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:10.533832 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b2b8f-0941-4674-abce-044a67b4f725 service nova] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:10.534266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b2b8f-0941-4674-abce-044a67b4f725 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:10.546891 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b2b8f-0941-4674-abce-044a67b4f725 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:10.552399 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b2b8f-0941-4674-abce-044a67b4f725 service nova] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:10.609382 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 894/1771] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:54:10.620361 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:10.807602 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9941752e-1350-4e69-81d8-d5a2f0512f8b service cinder] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:10.809460 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-9941752e-1350-4e69-81d8-d5a2f0512f8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:10.894199 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b2b8f-0941-4674-abce-044a67b4f725 service nova] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:10.917354 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b2b8f-0941-4674-abce-044a67b4f725 service nova] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:10.940972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39144b-1cf0-479a-87e0-69e4a7ed1958 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:10.941490 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39144b-1cf0-479a-87e0-69e4a7ed1958 service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:10.941888 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39144b-1cf0-479a-87e0-69e4a7ed1958 service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:10.942254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39144b-1cf0-479a-87e0-69e4a7ed1958 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:10.946622 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-4d39144b-1cf0-479a-87e0-69e4a7ed1958 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:11.036244 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9941752e-1350-4e69-81d8-d5a2f0512f8b service cinder] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:11.059361 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c46991-90f4-43cd-97e9-3f84a2f451e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:11.059361 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c46991-90f4-43cd-97e9-3f84a2f451e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:11.059361 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c46991-90f4-43cd-97e9-3f84a2f451e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:11.059361 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c46991-90f4-43cd-97e9-3f84a2f451e8 admin admin] PATH_INFO: `/v3/users/9635007b04174749ba05a6879ca87b8f` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:11.059775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c46991-90f4-43cd-97e9-3f84a2f451e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9635007b04174749ba05a6879ca87b8f)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:11.069643 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c46991-90f4-43cd-97e9-3f84a2f451e8 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:11.109738 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d39144b-1cf0-479a-87e0-69e4a7ed1958 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:11.111517 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d39144b-1cf0-479a-87e0-69e4a7ed1958 service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:11.137506 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 895/1772] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:54:11.188969 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868eaf1b-ddb7-4ba7-bbf9-896d5648e97b None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:11.189057 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868eaf1b-ddb7-4ba7-bbf9-896d5648e97b None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:11.189377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868eaf1b-ddb7-4ba7-bbf9-896d5648e97b None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:11.191728 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 896/1773] 10.222.0.59 () {54 vars in 921 bytes} [Fri Aug 26 00:54:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:54:11.300449 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d764f4-7f4c-4425-94f9-65101bef9b82 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:11.304692 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d764f4-7f4c-4425-94f9-65101bef9b82 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:11.304692 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d764f4-7f4c-4425-94f9-65101bef9b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:11.367951 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-18c46991-90f4-43cd-97e9-3f84a2f451e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9635007b04174749ba05a6879ca87b8f'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:11.395968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-64d764f4-7f4c-4425-94f9-65101bef9b82 None None] MFA Rules not processed for user `0602593263ca4b05a657d5b7697f932f`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:11.402174 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-64d764f4-7f4c-4425-94f9-65101bef9b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:11.424674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-18c46991-90f4-43cd-97e9-3f84a2f451e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9635007b04174749ba05a6879ca87b8f'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:11.446300 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 897/1774] 10.222.0.59 () {58 vars in 1018 bytes} [Fri Aug 26 00:54:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:54:11.446300 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d764f4-7f4c-4425-94f9-65101bef9b82 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:11.446791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-64d764f4-7f4c-4425-94f9-65101bef9b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:11.485685 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d764f4-7f4c-4425-94f9-65101bef9b82 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:11.499804 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3961855-42c6-4d52-ab1b-ae183a4b6600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:11.500516 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3961855-42c6-4d52-ab1b-ae183a4b6600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:11.500791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3961855-42c6-4d52-ab1b-ae183a4b6600 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:11.501070 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3961855-42c6-4d52-ab1b-ae183a4b6600 admin admin] PATH_INFO: `/v3/users/721215eaa8164f0c8a2049dd5581d89c` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:11.504455 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3961855-42c6-4d52-ab1b-ae183a4b6600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=721215eaa8164f0c8a2049dd5581d89c)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:11.511516 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3961855-42c6-4d52-ab1b-ae183a4b6600 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:11.512745 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-18c46991-90f4-43cd-97e9-3f84a2f451e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9635007b04174749ba05a6879ca87b8f'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:11.513479 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-18c46991-90f4-43cd-97e9-3f84a2f451e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9635007b04174749ba05a6879ca87b8f'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:11.514446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-18c46991-90f4-43cd-97e9-3f84a2f451e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9635007b04174749ba05a6879ca87b8f'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:11.554367 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 878/1775] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:54:10 2022] DELETE /identity/v3/users/9635007b04174749ba05a6879ca87b8f => generated 0 bytes in 748 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:11.559615 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0628a95-8ddf-4d73-9c48-0ce6d0d9cd33 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:11.559765 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0628a95-8ddf-4d73-9c48-0ce6d0d9cd33 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:11.560497 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0628a95-8ddf-4d73-9c48-0ce6d0d9cd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:11.574396 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:11.628361 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-c0628a95-8ddf-4d73-9c48-0ce6d0d9cd33 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:11.639624 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-c0628a95-8ddf-4d73-9c48-0ce6d0d9cd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:11.832671 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:11.841071 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-d3961855-42c6-4d52-ab1b-ae183a4b6600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '721215eaa8164f0c8a2049dd5581d89c'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:11.844071 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-d3961855-42c6-4d52-ab1b-ae183a4b6600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '721215eaa8164f0c8a2049dd5581d89c'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:11.877152 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-d3961855-42c6-4d52-ab1b-ae183a4b6600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '721215eaa8164f0c8a2049dd5581d89c'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:11.922658 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 879/1776] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:11.923662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0628a95-8ddf-4d73-9c48-0ce6d0d9cd33 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:11.928115 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-c0628a95-8ddf-4d73-9c48-0ce6d0d9cd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:11.979061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-d3961855-42c6-4d52-ab1b-ae183a4b6600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '721215eaa8164f0c8a2049dd5581d89c'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:11.980722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-d3961855-42c6-4d52-ab1b-ae183a4b6600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '721215eaa8164f0c8a2049dd5581d89c'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:12.014887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 898/1777] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:54:11 2022] DELETE /identity/v3/users/721215eaa8164f0c8a2049dd5581d89c => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:12.015915 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3961855-42c6-4d52-ab1b-ae183a4b6600 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:12.025206 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-d3961855-42c6-4d52-ab1b-ae183a4b6600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:12.124837 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0628a95-8ddf-4d73-9c48-0ce6d0d9cd33 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:12.133299 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3961855-42c6-4d52-ab1b-ae183a4b6600 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:12.137954 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60cd0b-e6ce-4083-8373-ef39f813fc79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:12.138804 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60cd0b-e6ce-4083-8373-ef39f813fc79 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:12.139248 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60cd0b-e6ce-4083-8373-ef39f813fc79 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:12.139984 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60cd0b-e6ce-4083-8373-ef39f813fc79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:12.222141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60cd0b-e6ce-4083-8373-ef39f813fc79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:12.225132 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60cd0b-e6ce-4083-8373-ef39f813fc79 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:12.238178 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 899/1778] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:54:12.240329 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828df53-13ae-42d9-af13-c96d8da41128 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:12.240890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828df53-13ae-42d9-af13-c96d8da41128 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:12.241218 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828df53-13ae-42d9-af13-c96d8da41128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:12.268699 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-9828df53-13ae-42d9-af13-c96d8da41128 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:12.270212 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-9828df53-13ae-42d9-af13-c96d8da41128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:12.288993 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 900/1779] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:12.310916 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828df53-13ae-42d9-af13-c96d8da41128 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:12.331464 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828df53-13ae-42d9-af13-c96d8da41128 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:12.355393 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f5665-a60a-4da9-b16a-4ed7af0dab41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:12.355393 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f5665-a60a-4da9-b16a-4ed7af0dab41 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:12.355393 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f5665-a60a-4da9-b16a-4ed7af0dab41 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:12.355393 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f5665-a60a-4da9-b16a-4ed7af0dab41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:12.355741 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860262db-3bdb-4e72-8c0b-a44a14af126e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:12.355741 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860262db-3bdb-4e72-8c0b-a44a14af126e service nova] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:12.355741 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860262db-3bdb-4e72-8c0b-a44a14af126e service nova] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:12.355741 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860262db-3bdb-4e72-8c0b-a44a14af126e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:12.366673 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-860262db-3bdb-4e72-8c0b-a44a14af126e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:12.375303 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f5665-a60a-4da9-b16a-4ed7af0dab41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:12.378979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f5665-a60a-4da9-b16a-4ed7af0dab41 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:12.398136 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 901/1780] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:54:12.411644 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860262db-3bdb-4e72-8c0b-a44a14af126e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:12.413387 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860262db-3bdb-4e72-8c0b-a44a14af126e service nova] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:12.445246 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 880/1781] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:54:12.557047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:12.877719 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:13.676702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:14.151429 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:14.371993 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f5665-a60a-4da9-b16a-4ed7af0dab41 service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:14.374598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-669f5665-a60a-4da9-b16a-4ed7af0dab41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:14.381045 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860262db-3bdb-4e72-8c0b-a44a14af126e service nova] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:14.382188 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-860262db-3bdb-4e72-8c0b-a44a14af126e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:14.418604 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860262db-3bdb-4e72-8c0b-a44a14af126e service nova] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:14.418962 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f5665-a60a-4da9-b16a-4ed7af0dab41 service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:14.428071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebe29f-bd77-4353-932e-156f807509e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:14.429174 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebe29f-bd77-4353-932e-156f807509e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:14.429677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebe29f-bd77-4353-932e-156f807509e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:14.430156 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebe29f-bd77-4353-932e-156f807509e0 admin admin] PATH_INFO: `/v3/projects/b17ed5a9456e48a29e8851fc66da69f5` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:14.431576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ebe29f-bd77-4353-932e-156f807509e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b17ed5a9456e48a29e8851fc66da69f5)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:14.435245 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ebe29f-bd77-4353-932e-156f807509e0 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:14.446521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4cb6a6-230f-4954-968c-9e0db442303b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:14.447216 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4cb6a6-230f-4954-968c-9e0db442303b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:14.447498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4cb6a6-230f-4954-968c-9e0db442303b admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:14.447759 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4cb6a6-230f-4954-968c-9e0db442303b admin admin] PATH_INFO: `/v3/projects/557a9c6f50d640f1af2e920b05715907` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:14.449168 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4cb6a6-230f-4954-968c-9e0db442303b admin admin] RBAC: Authorizing `identity:delete_project(project_id=557a9c6f50d640f1af2e920b05715907)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:14.452069 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4cb6a6-230f-4954-968c-9e0db442303b admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:14.623126 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-19ebe29f-bd77-4353-932e-156f807509e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b17ed5a9456e48a29e8851fc66da69f5', 'request_id': 'req-19ebe29f-bd77-4353-932e-156f807509e0'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:14.671979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-cb4cb6a6-230f-4954-968c-9e0db442303b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '557a9c6f50d640f1af2e920b05715907', 'request_id': 'req-cb4cb6a6-230f-4954-968c-9e0db442303b'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:14.724682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-19ebe29f-bd77-4353-932e-156f807509e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b17ed5a9456e48a29e8851fc66da69f5', 'request_id': 'req-19ebe29f-bd77-4353-932e-156f807509e0'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:14.747413 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-19ebe29f-bd77-4353-932e-156f807509e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b17ed5a9456e48a29e8851fc66da69f5', 'request_id': 'req-19ebe29f-bd77-4353-932e-156f807509e0'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:14.816393 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-cb4cb6a6-230f-4954-968c-9e0db442303b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '557a9c6f50d640f1af2e920b05715907', 'request_id': 'req-cb4cb6a6-230f-4954-968c-9e0db442303b'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:14.822628 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-19ebe29f-bd77-4353-932e-156f807509e0 admin admin] The token cache is being invalidate because project b17ed5a9456e48a29e8851fc66da69f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:54:14.823042 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-19ebe29f-bd77-4353-932e-156f807509e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:14.823805 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-19ebe29f-bd77-4353-932e-156f807509e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:14.825455 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 881/1782] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:54:14 2022] DELETE /identity/v3/projects/b17ed5a9456e48a29e8851fc66da69f5 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:54:14.831038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-cb4cb6a6-230f-4954-968c-9e0db442303b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '557a9c6f50d640f1af2e920b05715907', 'request_id': 'req-cb4cb6a6-230f-4954-968c-9e0db442303b'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:14.841086 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebe29f-bd77-4353-932e-156f807509e0 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:14.849870 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-19ebe29f-bd77-4353-932e-156f807509e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:14.859679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-cb4cb6a6-230f-4954-968c-9e0db442303b admin admin] The token cache is being invalidate because project 557a9c6f50d640f1af2e920b05715907 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:54:14.860120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-cb4cb6a6-230f-4954-968c-9e0db442303b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:14.861333 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-cb4cb6a6-230f-4954-968c-9e0db442303b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:14.864191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 902/1783] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:54:14 2022] DELETE /identity/v3/projects/557a9c6f50d640f1af2e920b05715907 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:14.875416 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4cb6a6-230f-4954-968c-9e0db442303b admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:14.882038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-cb4cb6a6-230f-4954-968c-9e0db442303b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:14.993129 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebe29f-bd77-4353-932e-156f807509e0 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:15.038886 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4cb6a6-230f-4954-968c-9e0db442303b admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:15.064864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b079f1-2289-4df8-b71a-8eb71956ce29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:15.064864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b079f1-2289-4df8-b71a-8eb71956ce29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:15.064864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b079f1-2289-4df8-b71a-8eb71956ce29 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:15.064864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b079f1-2289-4df8-b71a-8eb71956ce29 admin admin] PATH_INFO: `/v3/users/3636d5458aee4fd1af1ab993afa45d6d` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:15.065368 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b079f1-2289-4df8-b71a-8eb71956ce29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3636d5458aee4fd1af1ab993afa45d6d)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:15.072255 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b079f1-2289-4df8-b71a-8eb71956ce29 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:15.118190 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5adf9d-fb93-4541-b1b4-49a5de814615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:15.118845 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5adf9d-fb93-4541-b1b4-49a5de814615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:15.119124 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5adf9d-fb93-4541-b1b4-49a5de814615 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:15.119435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5adf9d-fb93-4541-b1b4-49a5de814615 admin admin] PATH_INFO: `/v3/users/88157f1e0abb4ff393a14efaf0b2e77c` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:15.138657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5adf9d-fb93-4541-b1b4-49a5de814615 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88157f1e0abb4ff393a14efaf0b2e77c)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:15.138657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5adf9d-fb93-4541-b1b4-49a5de814615 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:15.282802 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:15.420047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-4d5adf9d-fb93-4541-b1b4-49a5de814615 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88157f1e0abb4ff393a14efaf0b2e77c'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:15.425610 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-e8b079f1-2289-4df8-b71a-8eb71956ce29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3636d5458aee4fd1af1ab993afa45d6d'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:15.428281 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-e8b079f1-2289-4df8-b71a-8eb71956ce29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3636d5458aee4fd1af1ab993afa45d6d'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:15.447826 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:15.467389 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-4d5adf9d-fb93-4541-b1b4-49a5de814615 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88157f1e0abb4ff393a14efaf0b2e77c'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:15.493742 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-e8b079f1-2289-4df8-b71a-8eb71956ce29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3636d5458aee4fd1af1ab993afa45d6d'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:15.545373 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-4d5adf9d-fb93-4541-b1b4-49a5de814615 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88157f1e0abb4ff393a14efaf0b2e77c'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:15.546115 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-4d5adf9d-fb93-4541-b1b4-49a5de814615 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88157f1e0abb4ff393a14efaf0b2e77c'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:15.546802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-4d5adf9d-fb93-4541-b1b4-49a5de814615 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88157f1e0abb4ff393a14efaf0b2e77c'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:15.565163 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 882/1784] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:54:14 2022] DELETE /identity/v3/users/88157f1e0abb4ff393a14efaf0b2e77c => generated 0 bytes in 735 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:15.566371 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5adf9d-fb93-4541-b1b4-49a5de814615 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:15.568851 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-4d5adf9d-fb93-4541-b1b4-49a5de814615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:15.608206 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-e8b079f1-2289-4df8-b71a-8eb71956ce29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3636d5458aee4fd1af1ab993afa45d6d'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:15.609303 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-e8b079f1-2289-4df8-b71a-8eb71956ce29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3636d5458aee4fd1af1ab993afa45d6d'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:15.638605 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 903/1785] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:54:14 2022] DELETE /identity/v3/users/3636d5458aee4fd1af1ab993afa45d6d => generated 0 bytes in 764 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:15.701234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5adf9d-fb93-4541-b1b4-49a5de814615 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:15.774715 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:15.788675 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9bf32-0b55-4e89-a5df-19d2d50895e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:15.791574 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9bf32-0b55-4e89-a5df-19d2d50895e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:15.791865 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9bf32-0b55-4e89-a5df-19d2d50895e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:15.792393 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9bf32-0b55-4e89-a5df-19d2d50895e2 admin admin] PATH_INFO: `/v3/users/06cc2a4e266b450bae22fc66ab711be5` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:15.795877 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9bf32-0b55-4e89-a5df-19d2d50895e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06cc2a4e266b450bae22fc66ab711be5)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:15.798925 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9bf32-0b55-4e89-a5df-19d2d50895e2 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:16.040691 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-51b9bf32-0b55-4e89-a5df-19d2d50895e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06cc2a4e266b450bae22fc66ab711be5'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:16.084236 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-51b9bf32-0b55-4e89-a5df-19d2d50895e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06cc2a4e266b450bae22fc66ab711be5'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:16.147116 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-51b9bf32-0b55-4e89-a5df-19d2d50895e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06cc2a4e266b450bae22fc66ab711be5'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:16.147867 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-51b9bf32-0b55-4e89-a5df-19d2d50895e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06cc2a4e266b450bae22fc66ab711be5'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:16.149153 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-51b9bf32-0b55-4e89-a5df-19d2d50895e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06cc2a4e266b450bae22fc66ab711be5'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:16.199697 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 883/1786] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:54:15 2022] DELETE /identity/v3/users/06cc2a4e266b450bae22fc66ab711be5 => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:16.204424 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f324125-cf54-4987-af83-6a105569d099 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:16.204715 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f324125-cf54-4987-af83-6a105569d099 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:16.204995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f324125-cf54-4987-af83-6a105569d099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:16.235527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-3f324125-cf54-4987-af83-6a105569d099 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:16.237079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-3f324125-cf54-4987-af83-6a105569d099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:16.331405 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 904/1787] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:16.363311 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9bf32-0b55-4e89-a5df-19d2d50895e2 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:16.365636 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-51b9bf32-0b55-4e89-a5df-19d2d50895e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:16.523773 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9bf32-0b55-4e89-a5df-19d2d50895e2 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:16.528459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83782a51-fd92-4033-a57b-51e590d7bc3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:16.529575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83782a51-fd92-4033-a57b-51e590d7bc3e service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:16.530044 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83782a51-fd92-4033-a57b-51e590d7bc3e service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:16.530499 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83782a51-fd92-4033-a57b-51e590d7bc3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:16.564027 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83782a51-fd92-4033-a57b-51e590d7bc3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:16.568634 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83782a51-fd92-4033-a57b-51e590d7bc3e service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:16.590248 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 884/1788] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:54:16.883581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f324125-cf54-4987-af83-6a105569d099 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:16.886790 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-3f324125-cf54-4987-af83-6a105569d099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:16.888466 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83782a51-fd92-4033-a57b-51e590d7bc3e service neutron] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:16.890746 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-83782a51-fd92-4033-a57b-51e590d7bc3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:16.911992 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83782a51-fd92-4033-a57b-51e590d7bc3e service neutron] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:16.912457 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f324125-cf54-4987-af83-6a105569d099 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:16.914540 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054677e-c5c2-432d-8b8d-cf56a9672e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:16.915322 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054677e-c5c2-432d-8b8d-cf56a9672e81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:16.915678 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054677e-c5c2-432d-8b8d-cf56a9672e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:16.916043 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054677e-c5c2-432d-8b8d-cf56a9672e81 admin admin] PATH_INFO: `/v3/projects/7ec5fa515a094c6189157a2dc006863c` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:16.917398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054677e-c5c2-432d-8b8d-cf56a9672e81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ec5fa515a094c6189157a2dc006863c)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:16.919383 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054677e-c5c2-432d-8b8d-cf56a9672e81 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:16.925528 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb2e3b-3f48-4aa0-8e72-094720dcc470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:16.926676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb2e3b-3f48-4aa0-8e72-094720dcc470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:16.926970 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb2e3b-3f48-4aa0-8e72-094720dcc470 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:16.927248 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb2e3b-3f48-4aa0-8e72-094720dcc470 admin admin] PATH_INFO: `/v3/projects/c17cc30e568e450a89d24188a861a500` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:16.932133 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb2e3b-3f48-4aa0-8e72-094720dcc470 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c17cc30e568e450a89d24188a861a500)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:16.934411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb2e3b-3f48-4aa0-8e72-094720dcc470 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:17.092861 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-3fbb2e3b-3f48-4aa0-8e72-094720dcc470 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c17cc30e568e450a89d24188a861a500', 'request_id': 'req-3fbb2e3b-3f48-4aa0-8e72-094720dcc470'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:17.093077 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2054677e-c5c2-432d-8b8d-cf56a9672e81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ec5fa515a094c6189157a2dc006863c', 'request_id': 'req-2054677e-c5c2-432d-8b8d-cf56a9672e81'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:17.202526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2054677e-c5c2-432d-8b8d-cf56a9672e81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ec5fa515a094c6189157a2dc006863c', 'request_id': 'req-2054677e-c5c2-432d-8b8d-cf56a9672e81'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:17.209705 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2054677e-c5c2-432d-8b8d-cf56a9672e81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ec5fa515a094c6189157a2dc006863c', 'request_id': 'req-2054677e-c5c2-432d-8b8d-cf56a9672e81'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:17.220543 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2054677e-c5c2-432d-8b8d-cf56a9672e81 admin admin] The token cache is being invalidate because project 7ec5fa515a094c6189157a2dc006863c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:54:17.220841 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2054677e-c5c2-432d-8b8d-cf56a9672e81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:17.221527 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2054677e-c5c2-432d-8b8d-cf56a9672e81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:17.223066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 885/1789] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:54:16 2022] DELETE /identity/v3/projects/7ec5fa515a094c6189157a2dc006863c => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:17.224003 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054677e-c5c2-432d-8b8d-cf56a9672e81 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:17.225541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-2054677e-c5c2-432d-8b8d-cf56a9672e81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:17.291209 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054677e-c5c2-432d-8b8d-cf56a9672e81 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:17.344704 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb11f3-cfc4-4e4d-bd71-8dd2790c9053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:17.345341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb11f3-cfc4-4e4d-bd71-8dd2790c9053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:17.345623 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb11f3-cfc4-4e4d-bd71-8dd2790c9053 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:17.345898 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb11f3-cfc4-4e4d-bd71-8dd2790c9053 admin admin] PATH_INFO: `/v3/projects/2f306ac368f04a8eaa4b00e1c5d1ae23` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:17.347019 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb11f3-cfc4-4e4d-bd71-8dd2790c9053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f306ac368f04a8eaa4b00e1c5d1ae23)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:17.349493 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb11f3-cfc4-4e4d-bd71-8dd2790c9053 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:17.396505 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-3fbb2e3b-3f48-4aa0-8e72-094720dcc470 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c17cc30e568e450a89d24188a861a500', 'request_id': 'req-3fbb2e3b-3f48-4aa0-8e72-094720dcc470'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:17.403004 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-3fbb2e3b-3f48-4aa0-8e72-094720dcc470 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c17cc30e568e450a89d24188a861a500', 'request_id': 'req-3fbb2e3b-3f48-4aa0-8e72-094720dcc470'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:17.415288 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-3fbb2e3b-3f48-4aa0-8e72-094720dcc470 admin admin] The token cache is being invalidate because project c17cc30e568e450a89d24188a861a500 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:54:17.415581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-3fbb2e3b-3f48-4aa0-8e72-094720dcc470 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:17.416240 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-3fbb2e3b-3f48-4aa0-8e72-094720dcc470 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:17.417894 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 905/1790] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:54:16 2022] DELETE /identity/v3/projects/c17cc30e568e450a89d24188a861a500 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:17.419080 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa030c-af6a-42dc-9493-85190f1b80a1 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:17.419364 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa030c-af6a-42dc-9493-85190f1b80a1 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:17.419647 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa030c-af6a-42dc-9493-85190f1b80a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:17.439062 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-39fa030c-af6a-42dc-9493-85190f1b80a1 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:17.440285 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-39fa030c-af6a-42dc-9493-85190f1b80a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:17.454596 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 906/1791] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:17.454596 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d3b82-eecd-40e7-aca2-623e4cf73c1b None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:17.454596 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d3b82-eecd-40e7-aca2-623e4cf73c1b None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:17.454596 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d3b82-eecd-40e7-aca2-623e4cf73c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:17.470885 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-4f4d3b82-eecd-40e7-aca2-623e4cf73c1b None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:17.480503 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-4f4d3b82-eecd-40e7-aca2-623e4cf73c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:17.481458 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 907/1792] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:17.482666 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb2c87-3cd0-424c-8b34-5b5764162b64 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:17.482950 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb2c87-3cd0-424c-8b34-5b5764162b64 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:17.483227 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb2c87-3cd0-424c-8b34-5b5764162b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:17.503213 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-81cb2c87-3cd0-424c-8b34-5b5764162b64 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:17.504819 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-81cb2c87-3cd0-424c-8b34-5b5764162b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:17.521130 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-efdb11f3-cfc4-4e4d-bd71-8dd2790c9053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f306ac368f04a8eaa4b00e1c5d1ae23', 'request_id': 'req-efdb11f3-cfc4-4e4d-bd71-8dd2790c9053'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:17.566316 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 908/1793] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:17.567853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52299cf-f7dd-4e51-97c3-da218b65f78e None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:17.568309 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52299cf-f7dd-4e51-97c3-da218b65f78e None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:17.568623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52299cf-f7dd-4e51-97c3-da218b65f78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:17.587420 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-f52299cf-f7dd-4e51-97c3-da218b65f78e None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:17.588794 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f52299cf-f7dd-4e51-97c3-da218b65f78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:17.596783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 909/1794] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:17.597639 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52299cf-f7dd-4e51-97c3-da218b65f78e None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:17.607913 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52299cf-f7dd-4e51-97c3-da218b65f78e None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:17.612693 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b8fa4-d4e5-47ce-8934-99c9027c5b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:17.613302 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b8fa4-d4e5-47ce-8934-99c9027c5b61 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:17.613584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b8fa4-d4e5-47ce-8934-99c9027c5b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:17.613880 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b8fa4-d4e5-47ce-8934-99c9027c5b61 admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:17.614584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b8fa4-d4e5-47ce-8934-99c9027c5b61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:54:17.614873 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b8fa4-d4e5-47ce-8934-99c9027c5b61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:17.616527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b8fa4-d4e5-47ce-8934-99c9027c5b61 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:17.628395 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 910/1795] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:54:17.629222 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b8fa4-d4e5-47ce-8934-99c9027c5b61 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:17.638602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b8fa4-d4e5-47ce-8934-99c9027c5b61 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:17.643103 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c504f-8d8c-49da-8dfe-78f1bfacb373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:17.643672 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c504f-8d8c-49da-8dfe-78f1bfacb373 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:17.643957 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c504f-8d8c-49da-8dfe-78f1bfacb373 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:17.644233 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c504f-8d8c-49da-8dfe-78f1bfacb373 admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:17.644894 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c504f-8d8c-49da-8dfe-78f1bfacb373 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:54:17.645718 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c504f-8d8c-49da-8dfe-78f1bfacb373 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:17.647269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c504f-8d8c-49da-8dfe-78f1bfacb373 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:17.663180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 911/1796] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:54:17.663180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c504f-8d8c-49da-8dfe-78f1bfacb373 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:17.670352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c504f-8d8c-49da-8dfe-78f1bfacb373 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:17.675281 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998f035-a84e-417a-bbcb-622526628deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:17.675877 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998f035-a84e-417a-bbcb-622526628deb admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:17.676187 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998f035-a84e-417a-bbcb-622526628deb admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:17.676519 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998f035-a84e-417a-bbcb-622526628deb admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:17.677161 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998f035-a84e-417a-bbcb-622526628deb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:17.679063 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998f035-a84e-417a-bbcb-622526628deb admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:17.812548 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-efdb11f3-cfc4-4e4d-bd71-8dd2790c9053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f306ac368f04a8eaa4b00e1c5d1ae23', 'request_id': 'req-efdb11f3-cfc4-4e4d-bd71-8dd2790c9053'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:17.815225 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-efdb11f3-cfc4-4e4d-bd71-8dd2790c9053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f306ac368f04a8eaa4b00e1c5d1ae23', 'request_id': 'req-efdb11f3-cfc4-4e4d-bd71-8dd2790c9053'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:17.826289 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-efdb11f3-cfc4-4e4d-bd71-8dd2790c9053 admin admin] The token cache is being invalidate because project 2f306ac368f04a8eaa4b00e1c5d1ae23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:54:17.826593 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-efdb11f3-cfc4-4e4d-bd71-8dd2790c9053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:17.827260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-efdb11f3-cfc4-4e4d-bd71-8dd2790c9053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:17.828802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 886/1797] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:54:17 2022] DELETE /identity/v3/projects/2f306ac368f04a8eaa4b00e1c5d1ae23 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:17.829680 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb11f3-cfc4-4e4d-bd71-8dd2790c9053 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:17.831203 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-efdb11f3-cfc4-4e4d-bd71-8dd2790c9053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:17.844175 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb11f3-cfc4-4e4d-bd71-8dd2790c9053 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:17.849445 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcbae1-f3d5-48cf-95c9-ef6afbbdbe89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:17.850175 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcbae1-f3d5-48cf-95c9-ef6afbbdbe89 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:17.850394 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcbae1-f3d5-48cf-95c9-ef6afbbdbe89 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:17.850793 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcbae1-f3d5-48cf-95c9-ef6afbbdbe89 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:17.851426 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcbae1-f3d5-48cf-95c9-ef6afbbdbe89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:17.853373 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcbae1-f3d5-48cf-95c9-ef6afbbdbe89 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:17.892603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 912/1798] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:54:17 2022] POST /identity/v3/projects => generated 381 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:17.893466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998f035-a84e-417a-bbcb-622526628deb admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:17.896758 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-6998f035-a84e-417a-bbcb-622526628deb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:17.964784 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998f035-a84e-417a-bbcb-622526628deb admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:17.969956 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8e08d-049a-417c-897e-fe7fd8fe2443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:17.971476 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8e08d-049a-417c-897e-fe7fd8fe2443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:17.971476 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8e08d-049a-417c-897e-fe7fd8fe2443 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:17.971476 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8e08d-049a-417c-897e-fe7fd8fe2443 admin admin] PATH_INFO: `/v3/users/3f3a3d16822d43b3814c0fd4baba8fb1` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:17.972465 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e8e08d-049a-417c-897e-fe7fd8fe2443 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f3a3d16822d43b3814c0fd4baba8fb1)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:17.975424 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e8e08d-049a-417c-897e-fe7fd8fe2443 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:18.041204 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 887/1799] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:54:17 2022] POST /identity/v3/projects => generated 379 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:18.041204 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcbae1-f3d5-48cf-95c9-ef6afbbdbe89 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:18.041204 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-34fcbae1-f3d5-48cf-95c9-ef6afbbdbe89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:18.101899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcbae1-f3d5-48cf-95c9-ef6afbbdbe89 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:18.106878 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a203f-60b5-4b39-80c1-239b825e7f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:18.111327 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a203f-60b5-4b39-80c1-239b825e7f76 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:18.111327 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a203f-60b5-4b39-80c1-239b825e7f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:18.111327 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a203f-60b5-4b39-80c1-239b825e7f76 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:18.111327 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a203f-60b5-4b39-80c1-239b825e7f76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:18.111327 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a203f-60b5-4b39-80c1-239b825e7f76 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:18.198018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-61e8e08d-049a-417c-897e-fe7fd8fe2443 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f3a3d16822d43b3814c0fd4baba8fb1'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:18.198256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:18.198684 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-61e8e08d-049a-417c-897e-fe7fd8fe2443 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f3a3d16822d43b3814c0fd4baba8fb1'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:18.203935 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 888/1800] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:54:18 2022] POST /identity/v3/users => generated 291 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:18.204857 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a203f-60b5-4b39-80c1-239b825e7f76 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:18.206598 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-e70a203f-60b5-4b39-80c1-239b825e7f76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:18.212453 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-61e8e08d-049a-417c-897e-fe7fd8fe2443 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f3a3d16822d43b3814c0fd4baba8fb1'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:18.220034 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a203f-60b5-4b39-80c1-239b825e7f76 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:18.225301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a029c-1424-4b4e-b0ee-5d386576191e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:18.231064 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a029c-1424-4b4e-b0ee-5d386576191e admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:18.231064 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a029c-1424-4b4e-b0ee-5d386576191e admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:18.231064 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a029c-1424-4b4e-b0ee-5d386576191e admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:18.231064 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a029c-1424-4b4e-b0ee-5d386576191e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:18.231064 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a029c-1424-4b4e-b0ee-5d386576191e admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:18.281729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-61e8e08d-049a-417c-897e-fe7fd8fe2443 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f3a3d16822d43b3814c0fd4baba8fb1'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:18.282353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-61e8e08d-049a-417c-897e-fe7fd8fe2443 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f3a3d16822d43b3814c0fd4baba8fb1'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:18.296213 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 913/1801] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:54:17 2022] DELETE /identity/v3/users/3f3a3d16822d43b3814c0fd4baba8fb1 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:18.299267 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8e08d-049a-417c-897e-fe7fd8fe2443 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:18.299267 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-61e8e08d-049a-417c-897e-fe7fd8fe2443 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:18.318094 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 889/1802] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:54:18 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:18.322130 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a029c-1424-4b4e-b0ee-5d386576191e admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:18.338002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a029c-1424-4b4e-b0ee-5d386576191e admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:18.369999 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8e08d-049a-417c-897e-fe7fd8fe2443 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:18.377699 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac511d8-187c-420f-8c2b-f688a4b74747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:18.378372 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac511d8-187c-420f-8c2b-f688a4b74747 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:18.378653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac511d8-187c-420f-8c2b-f688a4b74747 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:18.378923 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac511d8-187c-420f-8c2b-f688a4b74747 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:18.379699 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac511d8-187c-420f-8c2b-f688a4b74747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:18.381396 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac511d8-187c-420f-8c2b-f688a4b74747 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:18.401151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 914/1803] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:18.408024 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac511d8-187c-420f-8c2b-f688a4b74747 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:18.419302 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac511d8-187c-420f-8c2b-f688a4b74747 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:18.424223 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d921ec6-d6e5-44da-be6c-7a8bccf43efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:18.424872 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d921ec6-d6e5-44da-be6c-7a8bccf43efd admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:18.425152 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d921ec6-d6e5-44da-be6c-7a8bccf43efd admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:18.425427 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d921ec6-d6e5-44da-be6c-7a8bccf43efd admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:18.426063 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d921ec6-d6e5-44da-be6c-7a8bccf43efd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:18.426622 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224217cc-de2c-4e08-8a04-ad4c572a4d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:18.427343 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224217cc-de2c-4e08-8a04-ad4c572a4d1a admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:18.427686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d921ec6-d6e5-44da-be6c-7a8bccf43efd admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:18.427781 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224217cc-de2c-4e08-8a04-ad4c572a4d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:18.428113 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224217cc-de2c-4e08-8a04-ad4c572a4d1a admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:18.429249 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224217cc-de2c-4e08-8a04-ad4c572a4d1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:18.431218 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224217cc-de2c-4e08-8a04-ad4c572a4d1a admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:18.441527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 915/1804] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:18.445818 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 890/1805] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:18.445986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d921ec6-d6e5-44da-be6c-7a8bccf43efd admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:18.447383 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224217cc-de2c-4e08-8a04-ad4c572a4d1a admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:18.456628 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d921ec6-d6e5-44da-be6c-7a8bccf43efd admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:18.458790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224217cc-de2c-4e08-8a04-ad4c572a4d1a admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:18.462206 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85ac42-1b4d-4288-a219-a0037701cc0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:18.462956 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85ac42-1b4d-4288-a219-a0037701cc0e admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:18.463245 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85ac42-1b4d-4288-a219-a0037701cc0e admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:18.463859 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85ac42-1b4d-4288-a219-a0037701cc0e admin admin] PATH_INFO: `/v3/projects/764ba3fa11e94ebc86f37d720ccb5277/users/1292051fe7a742e0afce37d8e0ebd030/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:18.467793 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72147938-f6ea-435d-98c5-de38fb8f2b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:18.467793 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72147938-f6ea-435d-98c5-de38fb8f2b4f admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:18.467793 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72147938-f6ea-435d-98c5-de38fb8f2b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:18.467793 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72147938-f6ea-435d-98c5-de38fb8f2b4f admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:18.468156 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72147938-f6ea-435d-98c5-de38fb8f2b4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:18.468805 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72147938-f6ea-435d-98c5-de38fb8f2b4f admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:18.480069 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85ac42-1b4d-4288-a219-a0037701cc0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=764ba3fa11e94ebc86f37d720ccb5277, user_id=1292051fe7a742e0afce37d8e0ebd030, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:18.483899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 891/1806] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:18.484049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85ac42-1b4d-4288-a219-a0037701cc0e admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:18.485208 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72147938-f6ea-435d-98c5-de38fb8f2b4f admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:18.497451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72147938-f6ea-435d-98c5-de38fb8f2b4f admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:18.504298 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a5488-96bc-4906-b9a4-02ec17b0c3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:18.505258 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a5488-96bc-4906-b9a4-02ec17b0c3c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:18.506167 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a5488-96bc-4906-b9a4-02ec17b0c3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:18.506446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a5488-96bc-4906-b9a4-02ec17b0c3c5 admin admin] PATH_INFO: `/v3/projects/40b7aefc942b4e84a3904f98aa60a248/users/abb97797858f473bade95ccc16b2f869/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:18.523729 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a5488-96bc-4906-b9a4-02ec17b0c3c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40b7aefc942b4e84a3904f98aa60a248, user_id=abb97797858f473bade95ccc16b2f869, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:18.526679 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a5488-96bc-4906-b9a4-02ec17b0c3c5 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:18.591067 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 916/1807] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:18 2022] PUT /identity/v3/projects/764ba3fa11e94ebc86f37d720ccb5277/users/1292051fe7a742e0afce37d8e0ebd030/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:54:18.601943 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ee83e-6da9-4e39-9990-da70c06792f4 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:18.602359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ee83e-6da9-4e39-9990-da70c06792f4 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:18.602675 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ee83e-6da9-4e39-9990-da70c06792f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:18.634929 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-147ee83e-6da9-4e39-9990-da70c06792f4 None None] MFA Rules not processed for user `1292051fe7a742e0afce37d8e0ebd030`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:18.636167 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-147ee83e-6da9-4e39-9990-da70c06792f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:18.638708 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 892/1808] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:18 2022] PUT /identity/v3/projects/40b7aefc942b4e84a3904f98aa60a248/users/abb97797858f473bade95ccc16b2f869/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:18.640918 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d73f45-d70d-420c-9bed-d866267a7f1f None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:18.641390 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d73f45-d70d-420c-9bed-d866267a7f1f None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:18.641816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d73f45-d70d-420c-9bed-d866267a7f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:18.682497 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-35d73f45-d70d-420c-9bed-d866267a7f1f None None] MFA Rules not processed for user `abb97797858f473bade95ccc16b2f869`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:18.684381 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-35d73f45-d70d-420c-9bed-d866267a7f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:18.716784 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 917/1809] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:18.721670 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ee83e-6da9-4e39-9990-da70c06792f4 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:18.754588 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 893/1810] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:18.759556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d73f45-d70d-420c-9bed-d866267a7f1f None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:18.794666 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ee83e-6da9-4e39-9990-da70c06792f4 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:18.800301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd4cc8-f415-49a1-9a41-205c07af401c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:18.802080 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd4cc8-f415-49a1-9a41-205c07af401c admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:18.806102 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd4cc8-f415-49a1-9a41-205c07af401c admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:18.806102 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd4cc8-f415-49a1-9a41-205c07af401c admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:18.806102 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd4cc8-f415-49a1-9a41-205c07af401c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:18.810430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd4cc8-f415-49a1-9a41-205c07af401c admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:18.825468 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:18.844705 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d73f45-d70d-420c-9bed-d866267a7f1f None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:18.853241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7cc40-ec23-4ea3-a033-2141f3878cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:18.855283 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7cc40-ec23-4ea3-a033-2141f3878cbb admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:18.855283 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7cc40-ec23-4ea3-a033-2141f3878cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:18.855283 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7cc40-ec23-4ea3-a033-2141f3878cbb admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:18.855283 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7cc40-ec23-4ea3-a033-2141f3878cbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:18.856991 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7cc40-ec23-4ea3-a033-2141f3878cbb admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:18.929383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 918/1811] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:54:18 2022] POST /identity/v3/projects => generated 381 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:54:18.930421 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd4cc8-f415-49a1-9a41-205c07af401c admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:18.934939 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-e9bd4cc8-f415-49a1-9a41-205c07af401c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:18.965570 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 894/1812] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:54:18 2022] POST /identity/v3/projects => generated 379 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:18.966723 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7cc40-ec23-4ea3-a033-2141f3878cbb admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:19.083838 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd4cc8-f415-49a1-9a41-205c07af401c admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:19.130503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7cc40-ec23-4ea3-a033-2141f3878cbb admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:19.137767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6ad2c-b626-4fb0-bd0c-b119f322a45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:19.138897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6ad2c-b626-4fb0-bd0c-b119f322a45a admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:19.139446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6ad2c-b626-4fb0-bd0c-b119f322a45a admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:19.139873 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6ad2c-b626-4fb0-bd0c-b119f322a45a admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:19.140688 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6ad2c-b626-4fb0-bd0c-b119f322a45a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:19.147717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6ad2c-b626-4fb0-bd0c-b119f322a45a admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:19.179520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5521f74b-5c1f-4295-b463-51dda2ab5272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:19.180174 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521f74b-5c1f-4295-b463-51dda2ab5272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:19.180458 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521f74b-5c1f-4295-b463-51dda2ab5272 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:19.180761 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521f74b-5c1f-4295-b463-51dda2ab5272 admin admin] PATH_INFO: `/v3/projects/0fd76bc578364a74970d940693e7c2aa` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:19.181967 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5521f74b-5c1f-4295-b463-51dda2ab5272 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fd76bc578364a74970d940693e7c2aa)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:19.184225 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5521f74b-5c1f-4295-b463-51dda2ab5272 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:19.249819 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 895/1813] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:54:18 2022] POST /identity/v3/users => generated 297 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:19.250609 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6ad2c-b626-4fb0-bd0c-b119f322a45a admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:19.283671 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6ad2c-b626-4fb0-bd0c-b119f322a45a admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:19.291989 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6d0be-1b09-4244-8c0e-eb435178376b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:19.291989 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6d0be-1b09-4244-8c0e-eb435178376b admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:19.291989 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6d0be-1b09-4244-8c0e-eb435178376b admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:19.291989 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6d0be-1b09-4244-8c0e-eb435178376b admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:19.292593 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6d0be-1b09-4244-8c0e-eb435178376b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:19.294365 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6d0be-1b09-4244-8c0e-eb435178376b admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:19.324075 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-5521f74b-5c1f-4295-b463-51dda2ab5272 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fd76bc578364a74970d940693e7c2aa', 'request_id': 'req-5521f74b-5c1f-4295-b463-51dda2ab5272'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:19.419846 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 896/1814] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:54:19 2022] POST /identity/v3/users => generated 290 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:19.420829 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6d0be-1b09-4244-8c0e-eb435178376b admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:19.430387 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-5521f74b-5c1f-4295-b463-51dda2ab5272 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fd76bc578364a74970d940693e7c2aa', 'request_id': 'req-5521f74b-5c1f-4295-b463-51dda2ab5272'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:19.430906 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6d0be-1b09-4244-8c0e-eb435178376b admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:19.438377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-5521f74b-5c1f-4295-b463-51dda2ab5272 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fd76bc578364a74970d940693e7c2aa', 'request_id': 'req-5521f74b-5c1f-4295-b463-51dda2ab5272'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:19.450507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-5521f74b-5c1f-4295-b463-51dda2ab5272 admin admin] The token cache is being invalidate because project 0fd76bc578364a74970d940693e7c2aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:54:19.450875 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-5521f74b-5c1f-4295-b463-51dda2ab5272 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:19.451607 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-5521f74b-5c1f-4295-b463-51dda2ab5272 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:19.453229 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 919/1815] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:54:18 2022] DELETE /identity/v3/projects/0fd76bc578364a74970d940693e7c2aa => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:19.454092 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5521f74b-5c1f-4295-b463-51dda2ab5272 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:19.455667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-5521f74b-5c1f-4295-b463-51dda2ab5272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:19.500761 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6d3f0-f2fb-4c1b-b86f-aec70fdd5e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:19.501545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6d3f0-f2fb-4c1b-b86f-aec70fdd5e72 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:19.501824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6d3f0-f2fb-4c1b-b86f-aec70fdd5e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:19.502094 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6d3f0-f2fb-4c1b-b86f-aec70fdd5e72 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:19.502890 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6d3f0-f2fb-4c1b-b86f-aec70fdd5e72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:19.505238 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6d3f0-f2fb-4c1b-b86f-aec70fdd5e72 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:19.515419 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 897/1816] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:19.516666 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19e18b-b072-444f-a388-0669c5584da4 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:19.516951 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19e18b-b072-444f-a388-0669c5584da4 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:19.517230 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19e18b-b072-444f-a388-0669c5584da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:19.534252 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5521f74b-5c1f-4295-b463-51dda2ab5272 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:19.545649 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b893208-7a99-48bc-b647-4a16933990d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:19.546470 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b893208-7a99-48bc-b647-4a16933990d4 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:19.546987 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b893208-7a99-48bc-b647-4a16933990d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:19.547528 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b893208-7a99-48bc-b647-4a16933990d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:19.548758 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b893208-7a99-48bc-b647-4a16933990d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:19.551320 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b893208-7a99-48bc-b647-4a16933990d4 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:19.552612 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-bd19e18b-b072-444f-a388-0669c5584da4 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:19.553874 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-bd19e18b-b072-444f-a388-0669c5584da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:19.563936 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 920/1818] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:19.565054 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b893208-7a99-48bc-b647-4a16933990d4 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:19.567927 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 898/1818] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:19.569041 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-9b893208-7a99-48bc-b647-4a16933990d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:19.569192 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19e18b-b072-444f-a388-0669c5584da4 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:19.581285 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19e18b-b072-444f-a388-0669c5584da4 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:19.585600 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b893208-7a99-48bc-b647-4a16933990d4 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:19.586801 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b69e41-fa1d-4412-aa40-e52c73f0cdc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:19.587432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b69e41-fa1d-4412-aa40-e52c73f0cdc2 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:19.587711 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b69e41-fa1d-4412-aa40-e52c73f0cdc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:19.587983 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b69e41-fa1d-4412-aa40-e52c73f0cdc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:19.588685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b69e41-fa1d-4412-aa40-e52c73f0cdc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:19.590322 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b69e41-fa1d-4412-aa40-e52c73f0cdc2 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:19.592375 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ebcd3-818b-431c-854e-0b962f06c931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:19.593292 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ebcd3-818b-431c-854e-0b962f06c931 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:19.593731 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ebcd3-818b-431c-854e-0b962f06c931 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:19.594160 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ebcd3-818b-431c-854e-0b962f06c931 admin admin] PATH_INFO: `/v3/projects/c9dbb8dbec0d4049bcae42ccf46d1b4c/users/c951670e4dc24e82affa7ff9df13b4ce/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:19.605137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 899/1819] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:19.606781 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16adbec-37e4-4f0b-831a-02e991e82a2a None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:19.607238 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16adbec-37e4-4f0b-831a-02e991e82a2a None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:19.607783 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16adbec-37e4-4f0b-831a-02e991e82a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:19.616657 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ebcd3-818b-431c-854e-0b962f06c931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9dbb8dbec0d4049bcae42ccf46d1b4c, user_id=c951670e4dc24e82affa7ff9df13b4ce, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:19.619564 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ebcd3-818b-431c-854e-0b962f06c931 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:19.630005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-d16adbec-37e4-4f0b-831a-02e991e82a2a None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:19.631234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-d16adbec-37e4-4f0b-831a-02e991e82a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:19.645029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 900/1820] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:19.645982 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16adbec-37e4-4f0b-831a-02e991e82a2a None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:19.666665 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16adbec-37e4-4f0b-831a-02e991e82a2a None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:19.672810 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332a20c-4131-4741-bbb9-be819cf3f1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:19.673565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332a20c-4131-4741-bbb9-be819cf3f1df admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:19.674128 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332a20c-4131-4741-bbb9-be819cf3f1df admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:19.674509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332a20c-4131-4741-bbb9-be819cf3f1df admin admin] PATH_INFO: `/v3/projects/76fdde78a2cb4b3c97bfa7a8d3648ae8/users/42f02801facf4d5181f226af5255f4e4/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:19.693260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332a20c-4131-4741-bbb9-be819cf3f1df admin admin] RBAC: Authorizing `identity:create_grant(project_id=76fdde78a2cb4b3c97bfa7a8d3648ae8, user_id=42f02801facf4d5181f226af5255f4e4, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:19.696129 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332a20c-4131-4741-bbb9-be819cf3f1df admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:20.105560 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 921/1821] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:19 2022] PUT /identity/v3/projects/c9dbb8dbec0d4049bcae42ccf46d1b4c/users/c951670e4dc24e82affa7ff9df13b4ce/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:20.108352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ebcd3-818b-431c-854e-0b962f06c931 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:20.128559 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 901/1822] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:19 2022] PUT /identity/v3/projects/76fdde78a2cb4b3c97bfa7a8d3648ae8/users/42f02801facf4d5181f226af5255f4e4/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:20.130503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332a20c-4131-4741-bbb9-be819cf3f1df admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:20.181560 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ebcd3-818b-431c-854e-0b962f06c931 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:20.228150 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332a20c-4131-4741-bbb9-be819cf3f1df admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:20.238537 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de72f0c-6892-4d2a-9e92-0e0c9eaa9fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:20.239836 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de72f0c-6892-4d2a-9e92-0e0c9eaa9fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:20.240162 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de72f0c-6892-4d2a-9e92-0e0c9eaa9fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:20.240856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de72f0c-6892-4d2a-9e92-0e0c9eaa9fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:20.241908 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de72f0c-6892-4d2a-9e92-0e0c9eaa9fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:20.243151 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de72f0c-6892-4d2a-9e92-0e0c9eaa9fb0 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:20.246882 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cdfef1-e4a5-48f6-a3bb-faca46cae3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:20.247573 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cdfef1-e4a5-48f6-a3bb-faca46cae3c9 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:20.248219 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cdfef1-e4a5-48f6-a3bb-faca46cae3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:20.250207 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cdfef1-e4a5-48f6-a3bb-faca46cae3c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:20.250874 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cdfef1-e4a5-48f6-a3bb-faca46cae3c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:54:20.251179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cdfef1-e4a5-48f6-a3bb-faca46cae3c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:20.253387 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cdfef1-e4a5-48f6-a3bb-faca46cae3c9 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:20.255786 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 902/1823] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:20 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:20.256817 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abb1c6-459a-422e-9686-e99954ca969f None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:20.263186 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abb1c6-459a-422e-9686-e99954ca969f None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:20.263186 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abb1c6-459a-422e-9686-e99954ca969f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:20.270280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 922/1824] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:54:20.271463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cdfef1-e4a5-48f6-a3bb-faca46cae3c9 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:20.281546 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cdfef1-e4a5-48f6-a3bb-faca46cae3c9 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:20.286488 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-e8abb1c6-459a-422e-9686-e99954ca969f None None] MFA Rules not processed for user `42f02801facf4d5181f226af5255f4e4`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:20.286875 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262137b-e630-4fd2-86e3-e5b4bf580c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:20.287592 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262137b-e630-4fd2-86e3-e5b4bf580c1f admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:20.287896 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262137b-e630-4fd2-86e3-e5b4bf580c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:20.288169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262137b-e630-4fd2-86e3-e5b4bf580c1f admin admin] PATH_INFO: `/v3/projects/c9dbb8dbec0d4049bcae42ccf46d1b4c/users/c951670e4dc24e82affa7ff9df13b4ce/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:20.288463 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-e8abb1c6-459a-422e-9686-e99954ca969f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:20.291816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262137b-e630-4fd2-86e3-e5b4bf580c1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9dbb8dbec0d4049bcae42ccf46d1b4c, user_id=c951670e4dc24e82affa7ff9df13b4ce, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:20.293028 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262137b-e630-4fd2-86e3-e5b4bf580c1f admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:20.304974 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 923/1825] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:20 2022] PUT /identity/v3/projects/c9dbb8dbec0d4049bcae42ccf46d1b4c/users/c951670e4dc24e82affa7ff9df13b4ce/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:20.306013 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262137b-e630-4fd2-86e3-e5b4bf580c1f admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:20.364727 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 903/1826] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:20.364727 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ce85c-9bd9-46dd-8179-9f2ef63198a9 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:20.364727 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ce85c-9bd9-46dd-8179-9f2ef63198a9 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:20.364727 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ce85c-9bd9-46dd-8179-9f2ef63198a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:20.388072 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262137b-e630-4fd2-86e3-e5b4bf580c1f admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:20.393864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb1aed-90f4-4e8b-b625-cb1f2b47dac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:20.394600 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb1aed-90f4-4e8b-b625-cb1f2b47dac4 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:20.395116 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb1aed-90f4-4e8b-b625-cb1f2b47dac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:20.395386 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb1aed-90f4-4e8b-b625-cb1f2b47dac4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:20.396132 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb1aed-90f4-4e8b-b625-cb1f2b47dac4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:20.396718 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-0f4ce85c-9bd9-46dd-8179-9f2ef63198a9 None None] MFA Rules not processed for user `c951670e4dc24e82affa7ff9df13b4ce`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:20.397912 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-0f4ce85c-9bd9-46dd-8179-9f2ef63198a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:20.398508 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb1aed-90f4-4e8b-b625-cb1f2b47dac4 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:20.471304 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 904/1827] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:20 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:20.472138 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4ce85c-9bd9-46dd-8179-9f2ef63198a9 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:20.482153 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4ce85c-9bd9-46dd-8179-9f2ef63198a9 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:20.486888 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ab6c3-3959-4507-b742-411893217c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:20.488845 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ab6c3-3959-4507-b742-411893217c41 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:20.488845 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ab6c3-3959-4507-b742-411893217c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:20.488845 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ab6c3-3959-4507-b742-411893217c41 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:20.488845 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ab6c3-3959-4507-b742-411893217c41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:20.490775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ab6c3-3959-4507-b742-411893217c41 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:20.493005 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 924/1828] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:54:20 2022] POST /identity/v3/projects => generated 407 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:20.493933 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb1aed-90f4-4e8b-b625-cb1f2b47dac4 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:20.495602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-9adb1aed-90f4-4e8b-b625-cb1f2b47dac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:20.553769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb1aed-90f4-4e8b-b625-cb1f2b47dac4 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:20.557425 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618aee7d-bd19-4cdd-a9a8-7cb8d9987203 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:20.558047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618aee7d-bd19-4cdd-a9a8-7cb8d9987203 service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:20.558349 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618aee7d-bd19-4cdd-a9a8-7cb8d9987203 service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:20.558620 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618aee7d-bd19-4cdd-a9a8-7cb8d9987203 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:20.580659 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 905/1829] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:54:20 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:20.581490 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ab6c3-3959-4507-b742-411893217c41 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:20.621485 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618aee7d-bd19-4cdd-a9a8-7cb8d9987203 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:20.622935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618aee7d-bd19-4cdd-a9a8-7cb8d9987203 service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:20.633498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 925/1830] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:20 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:54:20.636059 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618aee7d-bd19-4cdd-a9a8-7cb8d9987203 service cinder] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:20.642358 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:20.646516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ab6c3-3959-4507-b742-411893217c41 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:20.651963 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4f0e2-1417-435f-ac0d-b27a9e57148c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:20.663356 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4f0e2-1417-435f-ac0d-b27a9e57148c admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:20.663356 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4f0e2-1417-435f-ac0d-b27a9e57148c admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:20.663356 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4f0e2-1417-435f-ac0d-b27a9e57148c admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:20.663356 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a4f0e2-1417-435f-ac0d-b27a9e57148c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:20.663356 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a4f0e2-1417-435f-ac0d-b27a9e57148c admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:20.700110 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618aee7d-bd19-4cdd-a9a8-7cb8d9987203 service cinder] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:20.705127 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae15c9-d1b5-49df-853d-debcdb58a5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:20.705804 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae15c9-d1b5-49df-853d-debcdb58a5de admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:20.706084 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae15c9-d1b5-49df-853d-debcdb58a5de admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:20.706358 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae15c9-d1b5-49df-853d-debcdb58a5de admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:20.706998 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae15c9-d1b5-49df-853d-debcdb58a5de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:20.709599 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae15c9-d1b5-49df-853d-debcdb58a5de admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:20.836089 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 906/1831] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:54:20 2022] POST /identity/v3/users => generated 304 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:20.839612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 926/1832] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:54:20 2022] POST /identity/v3/users => generated 294 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:20.840342 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4f0e2-1417-435f-ac0d-b27a9e57148c admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:20.841810 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae15c9-d1b5-49df-853d-debcdb58a5de admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:20.859302 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4f0e2-1417-435f-ac0d-b27a9e57148c admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:20.867003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae15c9-d1b5-49df-853d-debcdb58a5de admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:20.870466 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fb26a-0c24-4ca8-b5db-fcc9410af080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:20.870466 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fb26a-0c24-4ca8-b5db-fcc9410af080 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:20.870466 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fb26a-0c24-4ca8-b5db-fcc9410af080 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:20.871140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fb26a-0c24-4ca8-b5db-fcc9410af080 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:20.871816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423fb26a-0c24-4ca8-b5db-fcc9410af080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:20.873650 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423fb26a-0c24-4ca8-b5db-fcc9410af080 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:20.880341 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703323f9-e32b-4f10-be05-4f3145d98298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:20.881124 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703323f9-e32b-4f10-be05-4f3145d98298 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:20.881424 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703323f9-e32b-4f10-be05-4f3145d98298 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:20.881683 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703323f9-e32b-4f10-be05-4f3145d98298 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:20.882344 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703323f9-e32b-4f10-be05-4f3145d98298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:20.884014 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703323f9-e32b-4f10-be05-4f3145d98298 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:20.902243 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 927/1833] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:20 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:20.905533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 907/1834] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:20 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:20.905977 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703323f9-e32b-4f10-be05-4f3145d98298 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:20.907226 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fb26a-0c24-4ca8-b5db-fcc9410af080 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:20.920847 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703323f9-e32b-4f10-be05-4f3145d98298 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:20.922871 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fb26a-0c24-4ca8-b5db-fcc9410af080 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:20.927044 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f29c84-3e6f-463f-ba8c-25c4a19f03d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:20.927981 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f29c84-3e6f-463f-ba8c-25c4a19f03d9 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:20.928293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d1eb6-576d-434e-9f7e-1d7ef246e0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:20.928713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f29c84-3e6f-463f-ba8c-25c4a19f03d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:20.929113 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d1eb6-576d-434e-9f7e-1d7ef246e0b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:20.929379 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f29c84-3e6f-463f-ba8c-25c4a19f03d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:20.929483 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d1eb6-576d-434e-9f7e-1d7ef246e0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:20.929848 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d1eb6-576d-434e-9f7e-1d7ef246e0b7 admin admin] PATH_INFO: `/v3/projects/59a97f5f356f4979a788fff284c8c80b/users/1c8b48d7af0545fa9ec3b47f4eeb7895/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:20.930513 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f29c84-3e6f-463f-ba8c-25c4a19f03d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:20.935298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f29c84-3e6f-463f-ba8c-25c4a19f03d9 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:20.948417 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 928/1835] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:20.953582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f29c84-3e6f-463f-ba8c-25c4a19f03d9 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:20.956160 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d1eb6-576d-434e-9f7e-1d7ef246e0b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59a97f5f356f4979a788fff284c8c80b, user_id=1c8b48d7af0545fa9ec3b47f4eeb7895, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:20.960882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d1eb6-576d-434e-9f7e-1d7ef246e0b7 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:20.965134 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f29c84-3e6f-463f-ba8c-25c4a19f03d9 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:20.970932 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bca864-7550-4901-bbe6-860bc896e940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:20.971471 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bca864-7550-4901-bbe6-860bc896e940 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:20.971754 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bca864-7550-4901-bbe6-860bc896e940 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:20.972030 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bca864-7550-4901-bbe6-860bc896e940 admin admin] PATH_INFO: `/v3/projects/f93d2a4f8f4f4e888f3949bba39b6e4e/users/680c44298ade46d6a2bd31049d8499db/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:20.987519 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bca864-7550-4901-bbe6-860bc896e940 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f93d2a4f8f4f4e888f3949bba39b6e4e, user_id=680c44298ade46d6a2bd31049d8499db, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:20.992923 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bca864-7550-4901-bbe6-860bc896e940 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:21.019155 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 908/1836] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:20 2022] PUT /identity/v3/projects/59a97f5f356f4979a788fff284c8c80b/users/1c8b48d7af0545fa9ec3b47f4eeb7895/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:21.022724 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d1eb6-576d-434e-9f7e-1d7ef246e0b7 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:21.073629 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 929/1837] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:20 2022] PUT /identity/v3/projects/f93d2a4f8f4f4e888f3949bba39b6e4e/users/680c44298ade46d6a2bd31049d8499db/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:21.087311 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d1eb6-576d-434e-9f7e-1d7ef246e0b7 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:21.089449 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a568bf-d806-4a91-b062-dab76163ef85 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:21.089828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a568bf-d806-4a91-b062-dab76163ef85 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:21.090161 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a568bf-d806-4a91-b062-dab76163ef85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:21.130309 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-e4a568bf-d806-4a91-b062-dab76163ef85 None None] MFA Rules not processed for user `680c44298ade46d6a2bd31049d8499db`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:21.131532 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-e4a568bf-d806-4a91-b062-dab76163ef85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:21.153153 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e27274-fea0-49a2-92eb-695f4cebde6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:21.153772 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e27274-fea0-49a2-92eb-695f4cebde6e admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:21.154112 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e27274-fea0-49a2-92eb-695f4cebde6e admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:21.154393 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e27274-fea0-49a2-92eb-695f4cebde6e admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:21.155239 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e27274-fea0-49a2-92eb-695f4cebde6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:21.157132 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e27274-fea0-49a2-92eb-695f4cebde6e admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:21.170415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 909/1838] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:21 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 00:54:21.174345 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e27274-fea0-49a2-92eb-695f4cebde6e admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:21.201267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e27274-fea0-49a2-92eb-695f4cebde6e admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:21.216438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8927bc-1ade-4d5e-81f6-998bce160160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:21.216596 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8927bc-1ade-4d5e-81f6-998bce160160 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:21.216853 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8927bc-1ade-4d5e-81f6-998bce160160 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:21.217339 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8927bc-1ade-4d5e-81f6-998bce160160 admin admin] PATH_INFO: `/v3/projects/59a97f5f356f4979a788fff284c8c80b/users/1c8b48d7af0545fa9ec3b47f4eeb7895/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:21.219756 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8927bc-1ade-4d5e-81f6-998bce160160 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59a97f5f356f4979a788fff284c8c80b, user_id=1c8b48d7af0545fa9ec3b47f4eeb7895, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:21.222567 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8927bc-1ade-4d5e-81f6-998bce160160 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:21.231543 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 930/1839] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:21 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:21.234164 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 910/1840] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:21 2022] PUT /identity/v3/projects/59a97f5f356f4979a788fff284c8c80b/users/1c8b48d7af0545fa9ec3b47f4eeb7895/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:54:21.237454 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a568bf-d806-4a91-b062-dab76163ef85 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:21.244305 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7030da4-0460-43df-a637-1dc62c5febdd None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:21.248647 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7030da4-0460-43df-a637-1dc62c5febdd None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:21.248647 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7030da4-0460-43df-a637-1dc62c5febdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:21.252327 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a568bf-d806-4a91-b062-dab76163ef85 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:21.315718 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:21.315910 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-f7030da4-0460-43df-a637-1dc62c5febdd None None] MFA Rules not processed for user `1c8b48d7af0545fa9ec3b47f4eeb7895`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:21.315910 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-f7030da4-0460-43df-a637-1dc62c5febdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:21.341594 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93c1c3-2478-4275-b60c-aba925ec7ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:21.343868 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93c1c3-2478-4275-b60c-aba925ec7ec4 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:21.343868 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93c1c3-2478-4275-b60c-aba925ec7ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:21.343868 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93c1c3-2478-4275-b60c-aba925ec7ec4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:21.343868 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93c1c3-2478-4275-b60c-aba925ec7ec4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:21.345603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93c1c3-2478-4275-b60c-aba925ec7ec4 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:21.362479 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 911/1841] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:21 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:21.374662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7030da4-0460-43df-a637-1dc62c5febdd None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:21.376032 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-f7030da4-0460-43df-a637-1dc62c5febdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:21.436542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 931/1842] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:54:21 2022] POST /identity/v3/projects => generated 409 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:21.436542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93c1c3-2478-4275-b60c-aba925ec7ec4 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:21.450214 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7030da4-0460-43df-a637-1dc62c5febdd None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:21.505607 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2cc823-ef37-4ff5-b1fd-324357a12c7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:21.506675 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cc823-ef37-4ff5-b1fd-324357a12c7a service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:21.507155 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cc823-ef37-4ff5-b1fd-324357a12c7a service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:21.507647 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cc823-ef37-4ff5-b1fd-324357a12c7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:21.507761 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93c1c3-2478-4275-b60c-aba925ec7ec4 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:21.512406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9493f3-3f6c-4114-b7dc-e6a843859424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:21.513079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9493f3-3f6c-4114-b7dc-e6a843859424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:21.513379 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9493f3-3f6c-4114-b7dc-e6a843859424 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:21.513647 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9493f3-3f6c-4114-b7dc-e6a843859424 admin admin] PATH_INFO: `/v3/users/1292051fe7a742e0afce37d8e0ebd030` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:21.514783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9493f3-3f6c-4114-b7dc-e6a843859424 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1292051fe7a742e0afce37d8e0ebd030)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:21.516881 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9493f3-3f6c-4114-b7dc-e6a843859424 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:21.575152 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2cc823-ef37-4ff5-b1fd-324357a12c7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:21.576660 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2cc823-ef37-4ff5-b1fd-324357a12c7a service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:21.586758 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 912/1843] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:21 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:54:21.587555 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2cc823-ef37-4ff5-b1fd-324357a12c7a service cinder] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:21.597294 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2cc823-ef37-4ff5-b1fd-324357a12c7a service cinder] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:21.602958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf00916-3e3d-40e3-a13f-cac948fc0a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:21.603915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf00916-3e3d-40e3-a13f-cac948fc0a0e admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:21.604306 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf00916-3e3d-40e3-a13f-cac948fc0a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:21.604787 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf00916-3e3d-40e3-a13f-cac948fc0a0e admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:21.605524 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf00916-3e3d-40e3-a13f-cac948fc0a0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:21.607552 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf00916-3e3d-40e3-a13f-cac948fc0a0e admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:21.707223 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-0b9493f3-3f6c-4114-b7dc-e6a843859424 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1292051fe7a742e0afce37d8e0ebd030'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:21.708386 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-0b9493f3-3f6c-4114-b7dc-e6a843859424 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1292051fe7a742e0afce37d8e0ebd030'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:21.719958 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-0b9493f3-3f6c-4114-b7dc-e6a843859424 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1292051fe7a742e0afce37d8e0ebd030'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:21.732755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 913/1844] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:54:21 2022] POST /identity/v3/users => generated 311 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:21.734047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf00916-3e3d-40e3-a13f-cac948fc0a0e admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:21.735693 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-fbf00916-3e3d-40e3-a13f-cac948fc0a0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:21.747723 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf00916-3e3d-40e3-a13f-cac948fc0a0e admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:21.752331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce1c2f-29c1-422b-9b15-c9b135ff4dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:21.752956 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce1c2f-29c1-422b-9b15-c9b135ff4dfb admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:21.753238 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce1c2f-29c1-422b-9b15-c9b135ff4dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:21.753507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce1c2f-29c1-422b-9b15-c9b135ff4dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:21.754143 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce1c2f-29c1-422b-9b15-c9b135ff4dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:21.756316 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce1c2f-29c1-422b-9b15-c9b135ff4dfb admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:21.766282 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 914/1845] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:21.767049 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce1c2f-29c1-422b-9b15-c9b135ff4dfb admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:21.769702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-f4ce1c2f-29c1-422b-9b15-c9b135ff4dfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:21.781905 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce1c2f-29c1-422b-9b15-c9b135ff4dfb admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:21.785571 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cb6b77-b52c-4c1f-8b9e-3dd2ce71431a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:21.786436 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb6b77-b52c-4c1f-8b9e-3dd2ce71431a service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:21.786711 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb6b77-b52c-4c1f-8b9e-3dd2ce71431a service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:21.786979 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb6b77-b52c-4c1f-8b9e-3dd2ce71431a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:21.788722 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-81cb6b77-b52c-4c1f-8b9e-3dd2ce71431a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:21.825146 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-0b9493f3-3f6c-4114-b7dc-e6a843859424 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1292051fe7a742e0afce37d8e0ebd030'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:21.825834 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-0b9493f3-3f6c-4114-b7dc-e6a843859424 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1292051fe7a742e0afce37d8e0ebd030'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:21.839704 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 932/1846] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:54:21 2022] DELETE /identity/v3/users/1292051fe7a742e0afce37d8e0ebd030 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:21.840685 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9493f3-3f6c-4114-b7dc-e6a843859424 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:21.850296 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cb6b77-b52c-4c1f-8b9e-3dd2ce71431a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:21.851787 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cb6b77-b52c-4c1f-8b9e-3dd2ce71431a service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:21.861515 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 915/1847] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:54:21.863178 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8df5b0-3fda-4213-8ffb-ff1015e196f0 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:21.863334 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8df5b0-3fda-4213-8ffb-ff1015e196f0 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:21.863700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8df5b0-3fda-4213-8ffb-ff1015e196f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:21.891556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-ce8df5b0-3fda-4213-8ffb-ff1015e196f0 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:21.891556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-ce8df5b0-3fda-4213-8ffb-ff1015e196f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:21.908168 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9493f3-3f6c-4114-b7dc-e6a843859424 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:21.919209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4918be-9469-4cb2-88fa-d1ca50a53072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:21.919209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4918be-9469-4cb2-88fa-d1ca50a53072 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:21.919209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4918be-9469-4cb2-88fa-d1ca50a53072 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:21.919209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4918be-9469-4cb2-88fa-d1ca50a53072 admin admin] PATH_INFO: `/v3/projects/c3de64cde16d464c8f4b1775c82fc4bd/users/1e188c777ed341168e7ab8ba15c972e7/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:21.931154 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4918be-9469-4cb2-88fa-d1ca50a53072 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3de64cde16d464c8f4b1775c82fc4bd, user_id=1e188c777ed341168e7ab8ba15c972e7, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:21.933871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4918be-9469-4cb2-88fa-d1ca50a53072 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:21.948000 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 916/1848] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:21.957914 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8df5b0-3fda-4213-8ffb-ff1015e196f0 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:21.959318 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-ce8df5b0-3fda-4213-8ffb-ff1015e196f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:22.023491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8df5b0-3fda-4213-8ffb-ff1015e196f0 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:22.023491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32900d-894b-4b06-a8e6-985e4fe3e92d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:22.023665 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32900d-894b-4b06-a8e6-985e4fe3e92d service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:22.023898 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32900d-894b-4b06-a8e6-985e4fe3e92d service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:22.024239 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32900d-894b-4b06-a8e6-985e4fe3e92d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:22.034623 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe32900d-894b-4b06-a8e6-985e4fe3e92d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:22.036749 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe32900d-894b-4b06-a8e6-985e4fe3e92d service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:22.042691 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:22.049440 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 933/1849] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:21 2022] PUT /identity/v3/projects/c3de64cde16d464c8f4b1775c82fc4bd/users/1e188c777ed341168e7ab8ba15c972e7/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:22.054465 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4918be-9469-4cb2-88fa-d1ca50a53072 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:22.055362 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 917/1850] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:54:22.131940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4918be-9469-4cb2-88fa-d1ca50a53072 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:22.139489 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923164c-496e-41a2-b111-9b22c9362981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:22.140409 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923164c-496e-41a2-b111-9b22c9362981 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:22.140967 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923164c-496e-41a2-b111-9b22c9362981 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:22.141390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923164c-496e-41a2-b111-9b22c9362981 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:22.142252 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923164c-496e-41a2-b111-9b22c9362981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:22.144696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923164c-496e-41a2-b111-9b22c9362981 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:22.160948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 934/1851] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 00:54:22.166884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32900d-894b-4b06-a8e6-985e4fe3e92d service neutron] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:22.182661 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32900d-894b-4b06-a8e6-985e4fe3e92d service neutron] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:22.202719 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e6470-7df2-4cab-a805-62bb84ae9f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:22.203866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e6470-7df2-4cab-a805-62bb84ae9f6b admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:22.204225 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e6470-7df2-4cab-a805-62bb84ae9f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:22.204533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e6470-7df2-4cab-a805-62bb84ae9f6b admin admin] PATH_INFO: `/v3/projects/c3de64cde16d464c8f4b1775c82fc4bd/users/1e188c777ed341168e7ab8ba15c972e7/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:22.208348 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e6470-7df2-4cab-a805-62bb84ae9f6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3de64cde16d464c8f4b1775c82fc4bd, user_id=1e188c777ed341168e7ab8ba15c972e7, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:22.211194 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e6470-7df2-4cab-a805-62bb84ae9f6b admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:22.225225 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 918/1852] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:22 2022] PUT /identity/v3/projects/c3de64cde16d464c8f4b1775c82fc4bd/users/1e188c777ed341168e7ab8ba15c972e7/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:22.235723 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c4abd0-c50c-45d4-854b-8c55e2db5219 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:22.236048 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c4abd0-c50c-45d4-854b-8c55e2db5219 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:22.236295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c4abd0-c50c-45d4-854b-8c55e2db5219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:22.264742 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-17c4abd0-c50c-45d4-854b-8c55e2db5219 None None] MFA Rules not processed for user `1e188c777ed341168e7ab8ba15c972e7`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:22.265871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-17c4abd0-c50c-45d4-854b-8c55e2db5219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:22.354869 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 935/1853] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:22 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:22.362176 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e6470-7df2-4cab-a805-62bb84ae9f6b admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:22.364044 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-df4e6470-7df2-4cab-a805-62bb84ae9f6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:22.457161 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e6470-7df2-4cab-a805-62bb84ae9f6b admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:22.462507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33e455-3ff1-44ee-85b4-54788e6e6789 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:22.463143 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33e455-3ff1-44ee-85b4-54788e6e6789 service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:22.463423 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33e455-3ff1-44ee-85b4-54788e6e6789 service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:22.463700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33e455-3ff1-44ee-85b4-54788e6e6789 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:22.477332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa33e455-3ff1-44ee-85b4-54788e6e6789 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:22.481548 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa33e455-3ff1-44ee-85b4-54788e6e6789 service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:22.497228 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 919/1854] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:22 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:54:23.049590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c4abd0-c50c-45d4-854b-8c55e2db5219 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:23.078589 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-17c4abd0-c50c-45d4-854b-8c55e2db5219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:23.123069 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c4abd0-c50c-45d4-854b-8c55e2db5219 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:23.127622 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d548831-e1a8-4755-a140-5d8d5d526351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:23.128267 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d548831-e1a8-4755-a140-5d8d5d526351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:23.128585 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d548831-e1a8-4755-a140-5d8d5d526351 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:23.128862 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d548831-e1a8-4755-a140-5d8d5d526351 admin admin] PATH_INFO: `/v3/projects/764ba3fa11e94ebc86f37d720ccb5277` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:23.130542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d548831-e1a8-4755-a140-5d8d5d526351 admin admin] RBAC: Authorizing `identity:delete_project(project_id=764ba3fa11e94ebc86f37d720ccb5277)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:23.132520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d548831-e1a8-4755-a140-5d8d5d526351 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:23.238700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33e455-3ff1-44ee-85b4-54788e6e6789 service cinder] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:23.247503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33e455-3ff1-44ee-85b4-54788e6e6789 service cinder] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:23.251135 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e8c85-03b8-48a3-b188-f1dfb6e8cfd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:23.251742 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e8c85-03b8-48a3-b188-f1dfb6e8cfd0 service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:23.252022 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e8c85-03b8-48a3-b188-f1dfb6e8cfd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:23.252309 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e8c85-03b8-48a3-b188-f1dfb6e8cfd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:23.254036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-f90e8c85-03b8-48a3-b188-f1dfb6e8cfd0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:23.360836 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e8c85-03b8-48a3-b188-f1dfb6e8cfd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:23.363519 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e8c85-03b8-48a3-b188-f1dfb6e8cfd0 service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:23.377269 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 920/1855] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:23 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:54:23.410408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-8d548831-e1a8-4755-a140-5d8d5d526351 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '764ba3fa11e94ebc86f37d720ccb5277', 'request_id': 'req-8d548831-e1a8-4755-a140-5d8d5d526351'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:23.555716 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-8d548831-e1a8-4755-a140-5d8d5d526351 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '764ba3fa11e94ebc86f37d720ccb5277', 'request_id': 'req-8d548831-e1a8-4755-a140-5d8d5d526351'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:23.562131 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-8d548831-e1a8-4755-a140-5d8d5d526351 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '764ba3fa11e94ebc86f37d720ccb5277', 'request_id': 'req-8d548831-e1a8-4755-a140-5d8d5d526351'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:23.579702 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-8d548831-e1a8-4755-a140-5d8d5d526351 admin admin] The token cache is being invalidate because project 764ba3fa11e94ebc86f37d720ccb5277 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:54:23.579702 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-8d548831-e1a8-4755-a140-5d8d5d526351 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:23.579702 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-8d548831-e1a8-4755-a140-5d8d5d526351 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:23.579702 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 936/1856] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:54:23 2022] DELETE /identity/v3/projects/764ba3fa11e94ebc86f37d720ccb5277 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:23.580999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e8c85-03b8-48a3-b188-f1dfb6e8cfd0 service cinder] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:23.591593 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e8c85-03b8-48a3-b188-f1dfb6e8cfd0 service cinder] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:23.647328 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d5d38-fdd3-44c4-93a2-35319fb2ab45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:23.647969 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d5d38-fdd3-44c4-93a2-35319fb2ab45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:23.648251 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d5d38-fdd3-44c4-93a2-35319fb2ab45 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:23.648556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d5d38-fdd3-44c4-93a2-35319fb2ab45 admin admin] PATH_INFO: `/v3/users/c951670e4dc24e82affa7ff9df13b4ce` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:23.649672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d5d38-fdd3-44c4-93a2-35319fb2ab45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c951670e4dc24e82affa7ff9df13b4ce)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:23.651616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d5d38-fdd3-44c4-93a2-35319fb2ab45 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:23.756589 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:23.907844 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-ac8d5d38-fdd3-44c4-93a2-35319fb2ab45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c951670e4dc24e82affa7ff9df13b4ce'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:23.920744 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-ac8d5d38-fdd3-44c4-93a2-35319fb2ab45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c951670e4dc24e82affa7ff9df13b4ce'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:24.036510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-ac8d5d38-fdd3-44c4-93a2-35319fb2ab45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c951670e4dc24e82affa7ff9df13b4ce'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:24.037160 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-ac8d5d38-fdd3-44c4-93a2-35319fb2ab45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c951670e4dc24e82affa7ff9df13b4ce'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:24.038018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-ac8d5d38-fdd3-44c4-93a2-35319fb2ab45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c951670e4dc24e82affa7ff9df13b4ce'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:24.056848 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 921/1857] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:54:23 2022] DELETE /identity/v3/users/c951670e4dc24e82affa7ff9df13b4ce => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:24.528318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d548831-e1a8-4755-a140-5d8d5d526351 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:24.529807 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-8d548831-e1a8-4755-a140-5d8d5d526351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:24.543655 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d548831-e1a8-4755-a140-5d8d5d526351 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:24.601142 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c11cc-11cd-4b08-a534-9c5661819842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:24.601701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c11cc-11cd-4b08-a534-9c5661819842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:24.601976 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c11cc-11cd-4b08-a534-9c5661819842 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:24.602251 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c11cc-11cd-4b08-a534-9c5661819842 admin admin] PATH_INFO: `/v3/projects/c9dbb8dbec0d4049bcae42ccf46d1b4c` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:24.603472 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c11cc-11cd-4b08-a534-9c5661819842 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9dbb8dbec0d4049bcae42ccf46d1b4c)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:24.605349 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c11cc-11cd-4b08-a534-9c5661819842 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:24.856568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2f8c11cc-11cd-4b08-a534-9c5661819842 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9dbb8dbec0d4049bcae42ccf46d1b4c', 'request_id': 'req-2f8c11cc-11cd-4b08-a534-9c5661819842'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:26.386953 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2f8c11cc-11cd-4b08-a534-9c5661819842 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9dbb8dbec0d4049bcae42ccf46d1b4c', 'request_id': 'req-2f8c11cc-11cd-4b08-a534-9c5661819842'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:26.397339 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2f8c11cc-11cd-4b08-a534-9c5661819842 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9dbb8dbec0d4049bcae42ccf46d1b4c', 'request_id': 'req-2f8c11cc-11cd-4b08-a534-9c5661819842'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:26.420003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2f8c11cc-11cd-4b08-a534-9c5661819842 admin admin] The token cache is being invalidate because project c9dbb8dbec0d4049bcae42ccf46d1b4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:54:26.420372 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2f8c11cc-11cd-4b08-a534-9c5661819842 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:26.421168 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2f8c11cc-11cd-4b08-a534-9c5661819842 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:26.425302 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 937/1858] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:54:24 2022] DELETE /identity/v3/projects/c9dbb8dbec0d4049bcae42ccf46d1b4c => generated 0 bytes in 1896 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:26.430019 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c91ae7-88f9-48a0-99b0-78059a9cd7b6 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:26.430876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c91ae7-88f9-48a0-99b0-78059a9cd7b6 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:26.430876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c91ae7-88f9-48a0-99b0-78059a9cd7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:26.456036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-d6c91ae7-88f9-48a0-99b0-78059a9cd7b6 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:26.457498 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-d6c91ae7-88f9-48a0-99b0-78059a9cd7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:26.550930 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 922/1859] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:54:26.559269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4eed8-2507-4722-b5cc-3e30573de6b7 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:26.559736 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4eed8-2507-4722-b5cc-3e30573de6b7 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:26.560087 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4eed8-2507-4722-b5cc-3e30573de6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:26.593872 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-78f4eed8-2507-4722-b5cc-3e30573de6b7 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:26.596235 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-78f4eed8-2507-4722-b5cc-3e30573de6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:26.596367 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:26.605384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 938/1860] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:26.609976 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c91ae7-88f9-48a0-99b0-78059a9cd7b6 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:26.621185 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c91ae7-88f9-48a0-99b0-78059a9cd7b6 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:26.626539 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0bf89-6e57-4a4a-b3fa-cdb8da335fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:26.629845 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0bf89-6e57-4a4a-b3fa-cdb8da335fcc admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:26.629845 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0bf89-6e57-4a4a-b3fa-cdb8da335fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:26.629845 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0bf89-6e57-4a4a-b3fa-cdb8da335fcc admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:26.629845 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0bf89-6e57-4a4a-b3fa-cdb8da335fcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:54:26.629845 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0bf89-6e57-4a4a-b3fa-cdb8da335fcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:26.630669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0bf89-6e57-4a4a-b3fa-cdb8da335fcc admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:26.644655 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 923/1861] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:54:26.648520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4eed8-2507-4722-b5cc-3e30573de6b7 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:26.662417 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4eed8-2507-4722-b5cc-3e30573de6b7 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:26.667320 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f071413-acb4-4661-b007-608db1a257bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:26.667974 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f071413-acb4-4661-b007-608db1a257bd admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:26.668257 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f071413-acb4-4661-b007-608db1a257bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:26.668564 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f071413-acb4-4661-b007-608db1a257bd admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:26.669216 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f071413-acb4-4661-b007-608db1a257bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:26.671292 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f071413-acb4-4661-b007-608db1a257bd admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:26.827169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 939/1862] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:54:26 2022] POST /identity/v3/projects => generated 373 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:26.831291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0bf89-6e57-4a4a-b3fa-cdb8da335fcc admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:26.844218 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0bf89-6e57-4a4a-b3fa-cdb8da335fcc admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:26.919935 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0335f3e-6b59-4770-a32a-dccbf30a2b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:26.920935 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0335f3e-6b59-4770-a32a-dccbf30a2b62 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:26.921149 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0335f3e-6b59-4770-a32a-dccbf30a2b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:26.921424 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0335f3e-6b59-4770-a32a-dccbf30a2b62 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:26.922066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0335f3e-6b59-4770-a32a-dccbf30a2b62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:26.924087 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0335f3e-6b59-4770-a32a-dccbf30a2b62 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:27.010190 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 924/1863] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:54:26 2022] POST /identity/v3/users => generated 287 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:27.014109 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f071413-acb4-4661-b007-608db1a257bd admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:27.025175 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f071413-acb4-4661-b007-608db1a257bd admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:27.029748 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487bb58-5d5f-46ea-b0c4-efe8cf2c2b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:27.030366 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487bb58-5d5f-46ea-b0c4-efe8cf2c2b19 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:27.030648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487bb58-5d5f-46ea-b0c4-efe8cf2c2b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:27.030924 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487bb58-5d5f-46ea-b0c4-efe8cf2c2b19 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:27.031574 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b487bb58-5d5f-46ea-b0c4-efe8cf2c2b19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:27.033462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b487bb58-5d5f-46ea-b0c4-efe8cf2c2b19 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:27.044383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 940/1864] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:27.064880 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0335f3e-6b59-4770-a32a-dccbf30a2b62 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:27.075659 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0335f3e-6b59-4770-a32a-dccbf30a2b62 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:27.080716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c4e6f-c7d4-4f9a-a01d-b7ac216a158c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:27.081468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c4e6f-c7d4-4f9a-a01d-b7ac216a158c admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:27.081819 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c4e6f-c7d4-4f9a-a01d-b7ac216a158c admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:27.082167 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c4e6f-c7d4-4f9a-a01d-b7ac216a158c admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:27.083545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c4e6f-c7d4-4f9a-a01d-b7ac216a158c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:27.085185 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c4e6f-c7d4-4f9a-a01d-b7ac216a158c admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:27.096942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 925/1865] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:27.101307 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487bb58-5d5f-46ea-b0c4-efe8cf2c2b19 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:27.112330 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487bb58-5d5f-46ea-b0c4-efe8cf2c2b19 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:27.117699 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a702c-b8d8-4fb3-8f94-13ebba6b8189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:27.118393 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a702c-b8d8-4fb3-8f94-13ebba6b8189 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:27.118672 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a702c-b8d8-4fb3-8f94-13ebba6b8189 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:27.118942 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a702c-b8d8-4fb3-8f94-13ebba6b8189 admin admin] PATH_INFO: `/v3/projects/3b12789051b44c58a7fecf889bae7f82/users/6d936006687947a2ab7983614446b96d/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:27.134142 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a702c-b8d8-4fb3-8f94-13ebba6b8189 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b12789051b44c58a7fecf889bae7f82, user_id=6d936006687947a2ab7983614446b96d, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:27.137082 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a702c-b8d8-4fb3-8f94-13ebba6b8189 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:27.237590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 941/1866] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:27 2022] PUT /identity/v3/projects/3b12789051b44c58a7fecf889bae7f82/users/6d936006687947a2ab7983614446b96d/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:27.248805 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc961c-a49e-458f-a2bc-8fccb7da8532 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:27.249686 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc961c-a49e-458f-a2bc-8fccb7da8532 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:27.249686 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc961c-a49e-458f-a2bc-8fccb7da8532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:27.291370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-b4dc961c-a49e-458f-a2bc-8fccb7da8532 None None] MFA Rules not processed for user `6d936006687947a2ab7983614446b96d`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:27.291370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-b4dc961c-a49e-458f-a2bc-8fccb7da8532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:27.362707 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 926/1867] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:54:27.448715 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a702c-b8d8-4fb3-8f94-13ebba6b8189 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:27.451011 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-933a702c-b8d8-4fb3-8f94-13ebba6b8189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:27.518549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a702c-b8d8-4fb3-8f94-13ebba6b8189 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:27.523086 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d3ff21-7570-4365-a031-e1d548f9d6cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:27.523715 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3ff21-7570-4365-a031-e1d548f9d6cc service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:27.524018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3ff21-7570-4365-a031-e1d548f9d6cc service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:27.524287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3ff21-7570-4365-a031-e1d548f9d6cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:27.532828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d3ff21-7570-4365-a031-e1d548f9d6cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:27.534253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d3ff21-7570-4365-a031-e1d548f9d6cc service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:27.547833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 942/1868] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:54:27.571485 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc961c-a49e-458f-a2bc-8fccb7da8532 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:27.634103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc961c-a49e-458f-a2bc-8fccb7da8532 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:27.638695 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aed6c8-fa3e-44dd-bfc3-e0571edaf180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:27.639400 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aed6c8-fa3e-44dd-bfc3-e0571edaf180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:27.639685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aed6c8-fa3e-44dd-bfc3-e0571edaf180 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:27.639961 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aed6c8-fa3e-44dd-bfc3-e0571edaf180 admin admin] PATH_INFO: `/v3/users/6d936006687947a2ab7983614446b96d` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:27.641095 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aed6c8-fa3e-44dd-bfc3-e0571edaf180 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d936006687947a2ab7983614446b96d)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:27.643035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aed6c8-fa3e-44dd-bfc3-e0571edaf180 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:28.053888 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-02aed6c8-fa3e-44dd-bfc3-e0571edaf180 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d936006687947a2ab7983614446b96d'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:28.055482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-02aed6c8-fa3e-44dd-bfc3-e0571edaf180 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d936006687947a2ab7983614446b96d'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:28.167925 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-02aed6c8-fa3e-44dd-bfc3-e0571edaf180 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d936006687947a2ab7983614446b96d'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:28.168711 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-02aed6c8-fa3e-44dd-bfc3-e0571edaf180 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d936006687947a2ab7983614446b96d'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:28.169388 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-02aed6c8-fa3e-44dd-bfc3-e0571edaf180 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d936006687947a2ab7983614446b96d'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:28.183050 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 927/1869] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:54:27 2022] DELETE /identity/v3/users/6d936006687947a2ab7983614446b96d => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:28.187037 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfaa4c5-7d61-48f5-a6bf-6026427219ee None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:28.187326 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfaa4c5-7d61-48f5-a6bf-6026427219ee None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:28.187626 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfaa4c5-7d61-48f5-a6bf-6026427219ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:28.206888 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-ddfaa4c5-7d61-48f5-a6bf-6026427219ee None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:28.208100 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-ddfaa4c5-7d61-48f5-a6bf-6026427219ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:28.270479 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 943/1870] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:54:28.280168 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aed6c8-fa3e-44dd-bfc3-e0571edaf180 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:28.281653 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-02aed6c8-fa3e-44dd-bfc3-e0571edaf180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:28.337532 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aed6c8-fa3e-44dd-bfc3-e0571edaf180 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:28.341078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e3eab-2c27-47b7-9b54-79c6b5d1c7b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:28.341678 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e3eab-2c27-47b7-9b54-79c6b5d1c7b4 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:28.341957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e3eab-2c27-47b7-9b54-79c6b5d1c7b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:28.342229 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e3eab-2c27-47b7-9b54-79c6b5d1c7b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:28.351578 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e3eab-2c27-47b7-9b54-79c6b5d1c7b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:28.353056 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e3eab-2c27-47b7-9b54-79c6b5d1c7b4 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:28.362564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 928/1871] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:54:28.380275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:28.797941 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfaa4c5-7d61-48f5-a6bf-6026427219ee None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:28.799518 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-ddfaa4c5-7d61-48f5-a6bf-6026427219ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:28.815650 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfaa4c5-7d61-48f5-a6bf-6026427219ee None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:28.821197 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57734cd3-d84c-4ced-a56c-21dc1c745a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:28.821879 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57734cd3-d84c-4ced-a56c-21dc1c745a68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:28.822161 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57734cd3-d84c-4ced-a56c-21dc1c745a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:28.822439 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57734cd3-d84c-4ced-a56c-21dc1c745a68 admin admin] PATH_INFO: `/v3/projects/3b12789051b44c58a7fecf889bae7f82` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:28.824093 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57734cd3-d84c-4ced-a56c-21dc1c745a68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b12789051b44c58a7fecf889bae7f82)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:28.825920 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57734cd3-d84c-4ced-a56c-21dc1c745a68 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:28.927710 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-57734cd3-d84c-4ced-a56c-21dc1c745a68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b12789051b44c58a7fecf889bae7f82', 'request_id': 'req-57734cd3-d84c-4ced-a56c-21dc1c745a68'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:28.979527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-57734cd3-d84c-4ced-a56c-21dc1c745a68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b12789051b44c58a7fecf889bae7f82', 'request_id': 'req-57734cd3-d84c-4ced-a56c-21dc1c745a68'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:28.988050 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-57734cd3-d84c-4ced-a56c-21dc1c745a68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b12789051b44c58a7fecf889bae7f82', 'request_id': 'req-57734cd3-d84c-4ced-a56c-21dc1c745a68'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:29.004852 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-57734cd3-d84c-4ced-a56c-21dc1c745a68 admin admin] The token cache is being invalidate because project 3b12789051b44c58a7fecf889bae7f82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:54:29.005194 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-57734cd3-d84c-4ced-a56c-21dc1c745a68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:29.005817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-57734cd3-d84c-4ced-a56c-21dc1c745a68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:29.007682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 944/1872] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:54:28 2022] DELETE /identity/v3/projects/3b12789051b44c58a7fecf889bae7f82 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:29.014854 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124a4cc-dcba-47ce-8a1b-314340a00d00 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:29.014854 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124a4cc-dcba-47ce-8a1b-314340a00d00 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:29.014854 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124a4cc-dcba-47ce-8a1b-314340a00d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:29.042214 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-1124a4cc-dcba-47ce-8a1b-314340a00d00 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:29.043448 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-1124a4cc-dcba-47ce-8a1b-314340a00d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:29.102859 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 929/1873] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:29.111589 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654879e-0651-44c4-b3fb-12bf1704cd76 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:29.111871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654879e-0651-44c4-b3fb-12bf1704cd76 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:29.112151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654879e-0651-44c4-b3fb-12bf1704cd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:29.116764 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=8 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:29.134605 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-e654879e-0651-44c4-b3fb-12bf1704cd76 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:29.136011 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-e654879e-0651-44c4-b3fb-12bf1704cd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:29.144105 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 945/1874] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:29.148752 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124a4cc-dcba-47ce-8a1b-314340a00d00 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:29.162299 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124a4cc-dcba-47ce-8a1b-314340a00d00 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:29.169166 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a814bd2-42fa-49b7-b3e5-62933a3ba2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:29.169166 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a814bd2-42fa-49b7-b3e5-62933a3ba2af admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:29.169166 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a814bd2-42fa-49b7-b3e5-62933a3ba2af admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:29.169166 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a814bd2-42fa-49b7-b3e5-62933a3ba2af admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:29.169842 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a814bd2-42fa-49b7-b3e5-62933a3ba2af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:54:29.170147 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a814bd2-42fa-49b7-b3e5-62933a3ba2af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:29.171889 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a814bd2-42fa-49b7-b3e5-62933a3ba2af admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:29.185620 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 930/1875] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:54:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:54:29.189723 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654879e-0651-44c4-b3fb-12bf1704cd76 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:29.199904 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654879e-0651-44c4-b3fb-12bf1704cd76 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:29.205382 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7159dbd-aec8-424b-a4a8-2faa233f3004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:29.206072 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7159dbd-aec8-424b-a4a8-2faa233f3004 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:29.206528 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7159dbd-aec8-424b-a4a8-2faa233f3004 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:29.207040 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7159dbd-aec8-424b-a4a8-2faa233f3004 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:29.207773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7159dbd-aec8-424b-a4a8-2faa233f3004 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:29.209839 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7159dbd-aec8-424b-a4a8-2faa233f3004 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:29.312126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 946/1876] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:54:29 2022] POST /identity/v3/projects => generated 377 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:29.316480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a814bd2-42fa-49b7-b3e5-62933a3ba2af admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:29.326508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a814bd2-42fa-49b7-b3e5-62933a3ba2af admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:29.382033 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a4d87-d792-463b-be9d-9f532069378e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:29.382927 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a4d87-d792-463b-be9d-9f532069378e admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:29.382927 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a4d87-d792-463b-be9d-9f532069378e admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:29.383195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a4d87-d792-463b-be9d-9f532069378e admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:29.383828 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630a4d87-d792-463b-be9d-9f532069378e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:29.384854 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:29.385770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630a4d87-d792-463b-be9d-9f532069378e admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:29.556662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 931/1877] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:54:29 2022] POST /identity/v3/users => generated 289 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:29.560842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7159dbd-aec8-424b-a4a8-2faa233f3004 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:29.571775 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7159dbd-aec8-424b-a4a8-2faa233f3004 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:29.576262 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c197e24-db49-4dd7-9f8e-57a616c5d78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:29.576888 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c197e24-db49-4dd7-9f8e-57a616c5d78c admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:29.577274 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c197e24-db49-4dd7-9f8e-57a616c5d78c admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:29.577561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c197e24-db49-4dd7-9f8e-57a616c5d78c admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:29.578206 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c197e24-db49-4dd7-9f8e-57a616c5d78c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:29.579801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c197e24-db49-4dd7-9f8e-57a616c5d78c admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:29.590354 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 947/1878] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:29.599818 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a4d87-d792-463b-be9d-9f532069378e admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:29.609790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a4d87-d792-463b-be9d-9f532069378e admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:29.615014 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5442727-9d67-4fcd-8ed2-cb3061f0fe8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:29.615553 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5442727-9d67-4fcd-8ed2-cb3061f0fe8e admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:29.615835 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5442727-9d67-4fcd-8ed2-cb3061f0fe8e admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:29.616108 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5442727-9d67-4fcd-8ed2-cb3061f0fe8e admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:29.616789 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5442727-9d67-4fcd-8ed2-cb3061f0fe8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:29.618598 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5442727-9d67-4fcd-8ed2-cb3061f0fe8e admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:29.630370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 932/1879] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:29.643629 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c197e24-db49-4dd7-9f8e-57a616c5d78c admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:29.653574 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c197e24-db49-4dd7-9f8e-57a616c5d78c admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:29.658033 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a588d-aa45-4340-8e0e-3ca3eefda0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:29.658684 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a588d-aa45-4340-8e0e-3ca3eefda0e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:29.658971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a588d-aa45-4340-8e0e-3ca3eefda0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:29.659247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a588d-aa45-4340-8e0e-3ca3eefda0e4 admin admin] PATH_INFO: `/v3/projects/6e673709310b4a67b96a10ce79d83bf9/users/769d0c4198dd43b3861649b72a9a86b1/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:29.674998 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145a588d-aa45-4340-8e0e-3ca3eefda0e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e673709310b4a67b96a10ce79d83bf9, user_id=769d0c4198dd43b3861649b72a9a86b1, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:29.686336 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145a588d-aa45-4340-8e0e-3ca3eefda0e4 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:29.826406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 948/1880] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:29 2022] PUT /identity/v3/projects/6e673709310b4a67b96a10ce79d83bf9/users/769d0c4198dd43b3861649b72a9a86b1/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:29.837762 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe463dc-0678-4d48-b18b-736ce975f7d8 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:29.838054 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe463dc-0678-4d48-b18b-736ce975f7d8 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:29.838347 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe463dc-0678-4d48-b18b-736ce975f7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:29.867659 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-fbe463dc-0678-4d48-b18b-736ce975f7d8 None None] MFA Rules not processed for user `769d0c4198dd43b3861649b72a9a86b1`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:29.869222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-fbe463dc-0678-4d48-b18b-736ce975f7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:29.932810 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 933/1881] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:29.945184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a588d-aa45-4340-8e0e-3ca3eefda0e4 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:29.946854 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-145a588d-aa45-4340-8e0e-3ca3eefda0e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:30.006770 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a588d-aa45-4340-8e0e-3ca3eefda0e4 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:30.010472 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf47fe3f-688b-4205-b441-a25b84feba4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:30.011081 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47fe3f-688b-4205-b441-a25b84feba4c service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:30.011405 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47fe3f-688b-4205-b441-a25b84feba4c service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:30.011680 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47fe3f-688b-4205-b441-a25b84feba4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:30.019717 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf47fe3f-688b-4205-b441-a25b84feba4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:30.021373 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf47fe3f-688b-4205-b441-a25b84feba4c service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:30.030293 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 949/1882] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:54:30.043537 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:30.121486 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:30.562705 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:30.939415 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:31.264308 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe463dc-0678-4d48-b18b-736ce975f7d8 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:31.265734 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-fbe463dc-0678-4d48-b18b-736ce975f7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:31.381479 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe463dc-0678-4d48-b18b-736ce975f7d8 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:31.390030 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3e426-96f8-40ca-8478-cbb04f51ffe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:31.390030 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3e426-96f8-40ca-8478-cbb04f51ffe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:31.390030 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3e426-96f8-40ca-8478-cbb04f51ffe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:31.390030 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3e426-96f8-40ca-8478-cbb04f51ffe7 admin admin] PATH_INFO: `/v3/users/abb97797858f473bade95ccc16b2f869` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:31.390385 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba3e426-96f8-40ca-8478-cbb04f51ffe7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abb97797858f473bade95ccc16b2f869)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:31.390637 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba3e426-96f8-40ca-8478-cbb04f51ffe7 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:31.419943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:31.635568 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-8ba3e426-96f8-40ca-8478-cbb04f51ffe7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abb97797858f473bade95ccc16b2f869'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:31.648353 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-8ba3e426-96f8-40ca-8478-cbb04f51ffe7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abb97797858f473bade95ccc16b2f869'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:31.734000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:31.767771 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-8ba3e426-96f8-40ca-8478-cbb04f51ffe7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abb97797858f473bade95ccc16b2f869'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:31.768758 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-8ba3e426-96f8-40ca-8478-cbb04f51ffe7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abb97797858f473bade95ccc16b2f869'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:31.769507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-8ba3e426-96f8-40ca-8478-cbb04f51ffe7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abb97797858f473bade95ccc16b2f869'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:31.784724 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 934/1883] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:54:31 2022] DELETE /identity/v3/users/abb97797858f473bade95ccc16b2f869 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:31.790503 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e02e23-1aaa-4a0f-bf57-727a9e80cf0c None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:31.791013 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e02e23-1aaa-4a0f-bf57-727a9e80cf0c None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:31.791409 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e02e23-1aaa-4a0f-bf57-727a9e80cf0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:31.812873 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-45e02e23-1aaa-4a0f-bf57-727a9e80cf0c None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:31.814249 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-45e02e23-1aaa-4a0f-bf57-727a9e80cf0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:31.882888 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 950/1884] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:54:31.896623 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3e426-96f8-40ca-8478-cbb04f51ffe7 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:31.898260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-8ba3e426-96f8-40ca-8478-cbb04f51ffe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:31.958731 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3e426-96f8-40ca-8478-cbb04f51ffe7 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:31.962625 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75cb067-8bfc-4006-a999-dacb8989f078 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:31.963230 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75cb067-8bfc-4006-a999-dacb8989f078 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:31.963504 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75cb067-8bfc-4006-a999-dacb8989f078 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:31.963773 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75cb067-8bfc-4006-a999-dacb8989f078 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:31.972773 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75cb067-8bfc-4006-a999-dacb8989f078 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:31.974772 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75cb067-8bfc-4006-a999-dacb8989f078 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:31.984612 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 935/1885] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:54:32.008033 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:32.196263 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:32.350993 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:32.486578 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e02e23-1aaa-4a0f-bf57-727a9e80cf0c None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:32.494622 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-45e02e23-1aaa-4a0f-bf57-727a9e80cf0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:32.511588 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:32.532615 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e02e23-1aaa-4a0f-bf57-727a9e80cf0c None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:32.541622 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf6e2d-97f3-4213-86b4-7bc0966b2414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:32.543208 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf6e2d-97f3-4213-86b4-7bc0966b2414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:32.544368 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf6e2d-97f3-4213-86b4-7bc0966b2414 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:32.544871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf6e2d-97f3-4213-86b4-7bc0966b2414 admin admin] PATH_INFO: `/v3/projects/40b7aefc942b4e84a3904f98aa60a248` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:32.546601 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf6e2d-97f3-4213-86b4-7bc0966b2414 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40b7aefc942b4e84a3904f98aa60a248)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:32.548946 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf6e2d-97f3-4213-86b4-7bc0966b2414 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:32.588680 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:32.693032 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-28bf6e2d-97f3-4213-86b4-7bc0966b2414 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40b7aefc942b4e84a3904f98aa60a248', 'request_id': 'req-28bf6e2d-97f3-4213-86b4-7bc0966b2414'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:32.809887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-28bf6e2d-97f3-4213-86b4-7bc0966b2414 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40b7aefc942b4e84a3904f98aa60a248', 'request_id': 'req-28bf6e2d-97f3-4213-86b4-7bc0966b2414'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:32.817414 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-28bf6e2d-97f3-4213-86b4-7bc0966b2414 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40b7aefc942b4e84a3904f98aa60a248', 'request_id': 'req-28bf6e2d-97f3-4213-86b4-7bc0966b2414'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:32.830274 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-28bf6e2d-97f3-4213-86b4-7bc0966b2414 admin admin] The token cache is being invalidate because project 40b7aefc942b4e84a3904f98aa60a248 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:54:32.830836 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-28bf6e2d-97f3-4213-86b4-7bc0966b2414 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:32.831593 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-28bf6e2d-97f3-4213-86b4-7bc0966b2414 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:32.833804 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 951/1886] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:54:32 2022] DELETE /identity/v3/projects/40b7aefc942b4e84a3904f98aa60a248 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:32.837968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75cb067-8bfc-4006-a999-dacb8989f078 service neutron] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:32.848853 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75cb067-8bfc-4006-a999-dacb8989f078 service neutron] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:32.896495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c88f4a-0872-4644-8186-9b6d20411c44 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:32.897126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c88f4a-0872-4644-8186-9b6d20411c44 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:32.897362 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c88f4a-0872-4644-8186-9b6d20411c44 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:32.901930 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 952/1887] 10.222.0.59 () {54 vars in 921 bytes} [Fri Aug 26 00:54:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:54:32.910142 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434aea06-001f-4119-8e47-b5571a5b6da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:32.911785 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434aea06-001f-4119-8e47-b5571a5b6da1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:32.913565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434aea06-001f-4119-8e47-b5571a5b6da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:32.915540 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434aea06-001f-4119-8e47-b5571a5b6da1 admin admin] PATH_INFO: `/v3/users/42f02801facf4d5181f226af5255f4e4` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:32.921461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434aea06-001f-4119-8e47-b5571a5b6da1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42f02801facf4d5181f226af5255f4e4)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:32.922075 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434aea06-001f-4119-8e47-b5571a5b6da1 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:32.924416 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff9c96-61b3-4857-8e91-f4392f28abc8 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:32.924848 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff9c96-61b3-4857-8e91-f4392f28abc8 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:32.925208 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff9c96-61b3-4857-8e91-f4392f28abc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:32.952265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-dbff9c96-61b3-4857-8e91-f4392f28abc8 None None] MFA Rules not processed for user `0602593263ca4b05a657d5b7697f932f`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:32.954801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-dbff9c96-61b3-4857-8e91-f4392f28abc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:33.029746 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:33.074510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 953/1888] 10.222.0.59 () {58 vars in 1018 bytes} [Fri Aug 26 00:54:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:54:33.082596 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:33.087842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff9c96-61b3-4857-8e91-f4392f28abc8 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:33.089232 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-dbff9c96-61b3-4857-8e91-f4392f28abc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:33.106139 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff9c96-61b3-4857-8e91-f4392f28abc8 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:33.111828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6417b4-8eb6-47cc-9cf3-031a66343afb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:33.112496 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6417b4-8eb6-47cc-9cf3-031a66343afb service nova] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:33.114538 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6417b4-8eb6-47cc-9cf3-031a66343afb service nova] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:33.114824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6417b4-8eb6-47cc-9cf3-031a66343afb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:33.123775 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6417b4-8eb6-47cc-9cf3-031a66343afb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:33.125386 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6417b4-8eb6-47cc-9cf3-031a66343afb service nova] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:33.136258 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 954/1889] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:54:33.186044 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-434aea06-001f-4119-8e47-b5571a5b6da1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42f02801facf4d5181f226af5255f4e4'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:33.197813 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-434aea06-001f-4119-8e47-b5571a5b6da1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42f02801facf4d5181f226af5255f4e4'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:33.270323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-434aea06-001f-4119-8e47-b5571a5b6da1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42f02801facf4d5181f226af5255f4e4'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:33.271262 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-434aea06-001f-4119-8e47-b5571a5b6da1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42f02801facf4d5181f226af5255f4e4'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:33.271951 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-434aea06-001f-4119-8e47-b5571a5b6da1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42f02801facf4d5181f226af5255f4e4'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:33.287541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 936/1890] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:54:32 2022] DELETE /identity/v3/users/42f02801facf4d5181f226af5255f4e4 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:33.773874 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6417b4-8eb6-47cc-9cf3-031a66343afb service nova] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:33.775990 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-dd6417b4-8eb6-47cc-9cf3-031a66343afb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:33.791601 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:33.793854 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6417b4-8eb6-47cc-9cf3-031a66343afb service nova] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:33.906171 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e260af9-eb55-425b-9442-8cf522fabf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:33.906907 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e260af9-eb55-425b-9442-8cf522fabf1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:33.907277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e260af9-eb55-425b-9442-8cf522fabf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:33.907617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e260af9-eb55-425b-9442-8cf522fabf1f admin admin] PATH_INFO: `/v3/projects/76fdde78a2cb4b3c97bfa7a8d3648ae8` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:33.908896 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e260af9-eb55-425b-9442-8cf522fabf1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=76fdde78a2cb4b3c97bfa7a8d3648ae8)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:33.911706 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e260af9-eb55-425b-9442-8cf522fabf1f admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:34.018988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2e260af9-eb55-425b-9442-8cf522fabf1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76fdde78a2cb4b3c97bfa7a8d3648ae8', 'request_id': 'req-2e260af9-eb55-425b-9442-8cf522fabf1f'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:34.092521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2e260af9-eb55-425b-9442-8cf522fabf1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76fdde78a2cb4b3c97bfa7a8d3648ae8', 'request_id': 'req-2e260af9-eb55-425b-9442-8cf522fabf1f'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:34.099532 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2e260af9-eb55-425b-9442-8cf522fabf1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76fdde78a2cb4b3c97bfa7a8d3648ae8', 'request_id': 'req-2e260af9-eb55-425b-9442-8cf522fabf1f'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:34.112781 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2e260af9-eb55-425b-9442-8cf522fabf1f admin admin] The token cache is being invalidate because project 76fdde78a2cb4b3c97bfa7a8d3648ae8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:54:34.113078 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2e260af9-eb55-425b-9442-8cf522fabf1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:34.114486 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2e260af9-eb55-425b-9442-8cf522fabf1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:34.116066 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 955/1891] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:54:33 2022] DELETE /identity/v3/projects/76fdde78a2cb4b3c97bfa7a8d3648ae8 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:34.120111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434aea06-001f-4119-8e47-b5571a5b6da1 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:34.121650 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-434aea06-001f-4119-8e47-b5571a5b6da1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:34.138657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434aea06-001f-4119-8e47-b5571a5b6da1 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:34.192756 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13250be-8c5e-4165-91c6-6287868d4165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:34.193401 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13250be-8c5e-4165-91c6-6287868d4165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:34.193828 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13250be-8c5e-4165-91c6-6287868d4165 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:34.194018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13250be-8c5e-4165-91c6-6287868d4165 admin admin] PATH_INFO: `/v3/users/1c8b48d7af0545fa9ec3b47f4eeb7895` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:34.195218 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13250be-8c5e-4165-91c6-6287868d4165 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c8b48d7af0545fa9ec3b47f4eeb7895)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:34.196657 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:34.197209 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13250be-8c5e-4165-91c6-6287868d4165 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:34.336052 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c13250be-8c5e-4165-91c6-6287868d4165 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c8b48d7af0545fa9ec3b47f4eeb7895'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:34.349232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c13250be-8c5e-4165-91c6-6287868d4165 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c8b48d7af0545fa9ec3b47f4eeb7895'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:34.418659 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c13250be-8c5e-4165-91c6-6287868d4165 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c8b48d7af0545fa9ec3b47f4eeb7895'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:34.419317 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c13250be-8c5e-4165-91c6-6287868d4165 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c8b48d7af0545fa9ec3b47f4eeb7895'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:34.419909 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c13250be-8c5e-4165-91c6-6287868d4165 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c8b48d7af0545fa9ec3b47f4eeb7895'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:34.435757 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 937/1892] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:54:34 2022] DELETE /identity/v3/users/1c8b48d7af0545fa9ec3b47f4eeb7895 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:34.645904 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:34.860215 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e260af9-eb55-425b-9442-8cf522fabf1f admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:34.861734 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-2e260af9-eb55-425b-9442-8cf522fabf1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:34.877745 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e260af9-eb55-425b-9442-8cf522fabf1f admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:34.937525 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62693b43-15b8-46d3-9a7c-c06859a0e4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:34.938518 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62693b43-15b8-46d3-9a7c-c06859a0e4d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:34.938518 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62693b43-15b8-46d3-9a7c-c06859a0e4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:34.938708 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62693b43-15b8-46d3-9a7c-c06859a0e4d8 admin admin] PATH_INFO: `/v3/projects/59a97f5f356f4979a788fff284c8c80b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:34.939923 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62693b43-15b8-46d3-9a7c-c06859a0e4d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59a97f5f356f4979a788fff284c8c80b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:34.941945 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62693b43-15b8-46d3-9a7c-c06859a0e4d8 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:35.057009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-62693b43-15b8-46d3-9a7c-c06859a0e4d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59a97f5f356f4979a788fff284c8c80b', 'request_id': 'req-62693b43-15b8-46d3-9a7c-c06859a0e4d8'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:35.116494 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:35.150404 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-62693b43-15b8-46d3-9a7c-c06859a0e4d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59a97f5f356f4979a788fff284c8c80b', 'request_id': 'req-62693b43-15b8-46d3-9a7c-c06859a0e4d8'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:35.156785 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-62693b43-15b8-46d3-9a7c-c06859a0e4d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59a97f5f356f4979a788fff284c8c80b', 'request_id': 'req-62693b43-15b8-46d3-9a7c-c06859a0e4d8'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:35.168329 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-62693b43-15b8-46d3-9a7c-c06859a0e4d8 admin admin] The token cache is being invalidate because project 59a97f5f356f4979a788fff284c8c80b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:54:35.168655 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-62693b43-15b8-46d3-9a7c-c06859a0e4d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:35.169434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-62693b43-15b8-46d3-9a7c-c06859a0e4d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:35.170917 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 956/1893] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:54:34 2022] DELETE /identity/v3/projects/59a97f5f356f4979a788fff284c8c80b => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:35.183565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4df83-80a9-48d3-95e9-81772b57321c None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:35.183565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4df83-80a9-48d3-95e9-81772b57321c None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:35.183565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4df83-80a9-48d3-95e9-81772b57321c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:35.199995 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-7dd4df83-80a9-48d3-95e9-81772b57321c None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:35.201269 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-7dd4df83-80a9-48d3-95e9-81772b57321c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:35.263095 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 938/1894] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:35.271264 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc744c8-7866-4561-a638-bd20330494ba None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:35.271628 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc744c8-7866-4561-a638-bd20330494ba None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:35.271974 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc744c8-7866-4561-a638-bd20330494ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:35.293126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-3fc744c8-7866-4561-a638-bd20330494ba None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:35.294348 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-3fc744c8-7866-4561-a638-bd20330494ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:35.302671 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 957/1895] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:54:35.307007 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4df83-80a9-48d3-95e9-81772b57321c None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:35.317487 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4df83-80a9-48d3-95e9-81772b57321c None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:35.322201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f9de4-7ed8-4582-a0fa-080311cfdec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:35.322814 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f9de4-7ed8-4582-a0fa-080311cfdec9 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:35.323103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f9de4-7ed8-4582-a0fa-080311cfdec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:35.323378 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f9de4-7ed8-4582-a0fa-080311cfdec9 admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:35.324030 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f9de4-7ed8-4582-a0fa-080311cfdec9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:54:35.324334 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f9de4-7ed8-4582-a0fa-080311cfdec9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:35.326268 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f9de4-7ed8-4582-a0fa-080311cfdec9 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:35.339064 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 939/1896] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:54:35.343617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc744c8-7866-4561-a638-bd20330494ba None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:35.353696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc744c8-7866-4561-a638-bd20330494ba None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:35.358780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7d61e-1a7e-45b9-90aa-31bc64f28590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:35.359801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7d61e-1a7e-45b9-90aa-31bc64f28590 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:35.359801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7d61e-1a7e-45b9-90aa-31bc64f28590 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:35.359987 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7d61e-1a7e-45b9-90aa-31bc64f28590 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:35.360685 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7d61e-1a7e-45b9-90aa-31bc64f28590 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:35.362623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7d61e-1a7e-45b9-90aa-31bc64f28590 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:35.411197 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:35.450864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 958/1897] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:54:35 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:35.451069 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f9de4-7ed8-4582-a0fa-080311cfdec9 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:35.470893 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f9de4-7ed8-4582-a0fa-080311cfdec9 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:35.527162 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ee962-d09b-4ce0-87b4-3457ca49b981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:35.527781 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ee962-d09b-4ce0-87b4-3457ca49b981 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:35.528069 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ee962-d09b-4ce0-87b4-3457ca49b981 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:35.528350 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ee962-d09b-4ce0-87b4-3457ca49b981 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:35.529009 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5ee962-d09b-4ce0-87b4-3457ca49b981 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:35.530926 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5ee962-d09b-4ce0-87b4-3457ca49b981 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:35.573518 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:35.653742 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 940/1898] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:54:35 2022] POST /identity/v3/users => generated 287 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:35.657992 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7d61e-1a7e-45b9-90aa-31bc64f28590 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:35.668529 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7d61e-1a7e-45b9-90aa-31bc64f28590 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:35.670787 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:35.673010 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a30e50-04ef-49bc-b35c-c5e91ddc4997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:35.673605 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a30e50-04ef-49bc-b35c-c5e91ddc4997 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:35.677476 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a30e50-04ef-49bc-b35c-c5e91ddc4997 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:35.677476 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a30e50-04ef-49bc-b35c-c5e91ddc4997 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:35.677476 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a30e50-04ef-49bc-b35c-c5e91ddc4997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:35.679759 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a30e50-04ef-49bc-b35c-c5e91ddc4997 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:35.692335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 959/1899] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:35.696260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ee962-d09b-4ce0-87b4-3457ca49b981 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:35.716276 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ee962-d09b-4ce0-87b4-3457ca49b981 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:35.722827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60fa3e-f99c-44f4-8e4c-85e155d202e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:35.723797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60fa3e-f99c-44f4-8e4c-85e155d202e9 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:35.724188 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60fa3e-f99c-44f4-8e4c-85e155d202e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:35.724484 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60fa3e-f99c-44f4-8e4c-85e155d202e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:35.729670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60fa3e-f99c-44f4-8e4c-85e155d202e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:35.729670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60fa3e-f99c-44f4-8e4c-85e155d202e9 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:35.744206 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a30e50-04ef-49bc-b35c-c5e91ddc4997 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:35.747979 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 941/1900] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:35.756157 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a30e50-04ef-49bc-b35c-c5e91ddc4997 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:35.763407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf89378-3e7c-4777-9390-ebd1aa158411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:35.763407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf89378-3e7c-4777-9390-ebd1aa158411 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:35.763407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf89378-3e7c-4777-9390-ebd1aa158411 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:35.763407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf89378-3e7c-4777-9390-ebd1aa158411 admin admin] PATH_INFO: `/v3/projects/e9346a3c8fcd4b169a2a1f55d8cee2b0/users/c535e265953c4604b536538705b8285d/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:35.802965 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf89378-3e7c-4777-9390-ebd1aa158411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9346a3c8fcd4b169a2a1f55d8cee2b0, user_id=c535e265953c4604b536538705b8285d, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:35.808107 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf89378-3e7c-4777-9390-ebd1aa158411 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:35.964565 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 960/1901] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:35 2022] PUT /identity/v3/projects/e9346a3c8fcd4b169a2a1f55d8cee2b0/users/c535e265953c4604b536538705b8285d/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:54:35.969521 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d26bc-a99a-4d1c-a793-e41855aaf966 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:35.969812 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d26bc-a99a-4d1c-a793-e41855aaf966 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:35.970090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d26bc-a99a-4d1c-a793-e41855aaf966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:35.990858 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-b16d26bc-a99a-4d1c-a793-e41855aaf966 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:35.992181 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-b16d26bc-a99a-4d1c-a793-e41855aaf966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:36.060053 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 942/1902] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:54:36.071641 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf89378-3e7c-4777-9390-ebd1aa158411 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:36.073554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-2cf89378-3e7c-4777-9390-ebd1aa158411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:36.152792 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf89378-3e7c-4777-9390-ebd1aa158411 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:36.156849 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad434b-a08c-473d-b3d1-fbe9bde40658 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:36.157713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad434b-a08c-473d-b3d1-fbe9bde40658 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:36.157996 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad434b-a08c-473d-b3d1-fbe9bde40658 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:36.158273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad434b-a08c-473d-b3d1-fbe9bde40658 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:36.168775 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad434b-a08c-473d-b3d1-fbe9bde40658 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:36.170499 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad434b-a08c-473d-b3d1-fbe9bde40658 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:36.181821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:36.184156 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 961/1903] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:54:36.231292 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:36.738482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:37.219601 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d26bc-a99a-4d1c-a793-e41855aaf966 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:37.221594 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-b16d26bc-a99a-4d1c-a793-e41855aaf966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:37.236015 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d26bc-a99a-4d1c-a793-e41855aaf966 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:37.243802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f1e9a-61d9-4d91-95b8-a27f62d0c5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:37.244966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f1e9a-61d9-4d91-95b8-a27f62d0c5b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:37.245483 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f1e9a-61d9-4d91-95b8-a27f62d0c5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:37.246521 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f1e9a-61d9-4d91-95b8-a27f62d0c5b3 admin admin] PATH_INFO: `/v3/users/680c44298ade46d6a2bd31049d8499db` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:37.248393 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f1e9a-61d9-4d91-95b8-a27f62d0c5b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=680c44298ade46d6a2bd31049d8499db)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:37.250898 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f1e9a-61d9-4d91-95b8-a27f62d0c5b3 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:37.503515 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2a1f1e9a-61d9-4d91-95b8-a27f62d0c5b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '680c44298ade46d6a2bd31049d8499db'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:37.516332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2a1f1e9a-61d9-4d91-95b8-a27f62d0c5b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '680c44298ade46d6a2bd31049d8499db'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:37.579576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2a1f1e9a-61d9-4d91-95b8-a27f62d0c5b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '680c44298ade46d6a2bd31049d8499db'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:37.581654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2a1f1e9a-61d9-4d91-95b8-a27f62d0c5b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '680c44298ade46d6a2bd31049d8499db'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:37.583670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2a1f1e9a-61d9-4d91-95b8-a27f62d0c5b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '680c44298ade46d6a2bd31049d8499db'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:37.599385 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 943/1904] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:54:37 2022] DELETE /identity/v3/users/680c44298ade46d6a2bd31049d8499db => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:37.604140 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18410f78-0b90-4196-b089-06af3831018b None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:37.604626 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18410f78-0b90-4196-b089-06af3831018b None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:37.604801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18410f78-0b90-4196-b089-06af3831018b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:37.625007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-18410f78-0b90-4196-b089-06af3831018b None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:37.626225 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-18410f78-0b90-4196-b089-06af3831018b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:37.693411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 962/1905] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:54:37.703182 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f1e9a-61d9-4d91-95b8-a27f62d0c5b3 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:37.704622 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-2a1f1e9a-61d9-4d91-95b8-a27f62d0c5b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:37.791804 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f1e9a-61d9-4d91-95b8-a27f62d0c5b3 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:37.795562 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15513b06-0c6b-4fc2-b48b-41d133c315d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:37.796179 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15513b06-0c6b-4fc2-b48b-41d133c315d1 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:37.796462 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15513b06-0c6b-4fc2-b48b-41d133c315d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:37.796770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15513b06-0c6b-4fc2-b48b-41d133c315d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:37.807078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15513b06-0c6b-4fc2-b48b-41d133c315d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:37.810947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15513b06-0c6b-4fc2-b48b-41d133c315d1 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:37.819845 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 944/1906] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:54:38.005230 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3cb1e-6749-418e-9bdd-8e9a71985233 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:38.005547 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3cb1e-6749-418e-9bdd-8e9a71985233 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:38.005826 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3cb1e-6749-418e-9bdd-8e9a71985233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:38.036722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-76e3cb1e-6749-418e-9bdd-8e9a71985233 None None] MFA Rules not processed for user `c535e265953c4604b536538705b8285d`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:38.038188 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-76e3cb1e-6749-418e-9bdd-8e9a71985233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:38.107942 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 963/1907] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:38.112454 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15513b06-0c6b-4fc2-b48b-41d133c315d1 service neutron] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:38.114826 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-15513b06-0c6b-4fc2-b48b-41d133c315d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:38.136164 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15513b06-0c6b-4fc2-b48b-41d133c315d1 service neutron] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:38.142141 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8a5da-ca1b-44b3-96b3-f7f3e851e4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:38.143656 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8a5da-ca1b-44b3-96b3-f7f3e851e4b7 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:38.143777 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8a5da-ca1b-44b3-96b3-f7f3e851e4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:38.143979 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8a5da-ca1b-44b3-96b3-f7f3e851e4b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:38.144723 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d8a5da-ca1b-44b3-96b3-f7f3e851e4b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:38.146720 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d8a5da-ca1b-44b3-96b3-f7f3e851e4b7 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:38.260079 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:38.267507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 945/1908] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:54:38 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:38.271709 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e3cb1e-6749-418e-9bdd-8e9a71985233 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:38.283846 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e3cb1e-6749-418e-9bdd-8e9a71985233 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:38.341190 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:38.350810 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be6a46-c004-4ef8-9bc8-d3b8c8148462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:38.351485 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be6a46-c004-4ef8-9bc8-d3b8c8148462 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:38.351767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be6a46-c004-4ef8-9bc8-d3b8c8148462 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:38.352049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be6a46-c004-4ef8-9bc8-d3b8c8148462 admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:38.352733 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43be6a46-c004-4ef8-9bc8-d3b8c8148462 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:38.354942 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43be6a46-c004-4ef8-9bc8-d3b8c8148462 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:38.404193 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8a5da-ca1b-44b3-96b3-f7f3e851e4b7 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:38.406259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-00d8a5da-ca1b-44b3-96b3-f7f3e851e4b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:38.423055 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8a5da-ca1b-44b3-96b3-f7f3e851e4b7 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:38.428199 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cffffe-e4b5-4284-b1eb-a955b894735e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:38.428880 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cffffe-e4b5-4284-b1eb-a955b894735e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:38.429162 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cffffe-e4b5-4284-b1eb-a955b894735e admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:38.429436 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cffffe-e4b5-4284-b1eb-a955b894735e admin admin] PATH_INFO: `/v3/projects/f93d2a4f8f4f4e888f3949bba39b6e4e` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:38.436839 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cffffe-e4b5-4284-b1eb-a955b894735e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f93d2a4f8f4f4e888f3949bba39b6e4e)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:38.438841 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cffffe-e4b5-4284-b1eb-a955b894735e admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:38.489771 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 964/1909] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:54:38 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:38.493452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be6a46-c004-4ef8-9bc8-d3b8c8148462 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:38.504995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be6a46-c004-4ef8-9bc8-d3b8c8148462 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:38.511512 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17f423-5da5-41ee-b08e-db727b50678a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:38.512415 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17f423-5da5-41ee-b08e-db727b50678a admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:38.512939 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17f423-5da5-41ee-b08e-db727b50678a admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:38.513377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17f423-5da5-41ee-b08e-db727b50678a admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:38.514087 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17f423-5da5-41ee-b08e-db727b50678a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:38.515756 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17f423-5da5-41ee-b08e-db727b50678a admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:38.526822 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 965/1910] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 00:54:38.530807 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17f423-5da5-41ee-b08e-db727b50678a admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:38.540563 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17f423-5da5-41ee-b08e-db727b50678a admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:38.545484 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbde16-268b-4084-aefd-55ac7a10fd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:38.546137 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbde16-268b-4084-aefd-55ac7a10fd14 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:38.546422 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbde16-268b-4084-aefd-55ac7a10fd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:38.546703 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbde16-268b-4084-aefd-55ac7a10fd14 admin admin] PATH_INFO: `/v3/projects/4c5b35039a1846b8862f2f17449f18b9/users/85778cd9234949fb8a8cecb9b01320a5/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:38.561622 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcbde16-268b-4084-aefd-55ac7a10fd14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c5b35039a1846b8862f2f17449f18b9, user_id=85778cd9234949fb8a8cecb9b01320a5, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:38.564289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcbde16-268b-4084-aefd-55ac7a10fd14 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:38.694229 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 966/1911] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:38 2022] PUT /identity/v3/projects/4c5b35039a1846b8862f2f17449f18b9/users/85778cd9234949fb8a8cecb9b01320a5/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:54:38.699606 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbde16-268b-4084-aefd-55ac7a10fd14 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:38.705807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f3cffffe-e4b5-4284-b1eb-a955b894735e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f93d2a4f8f4f4e888f3949bba39b6e4e', 'request_id': 'req-f3cffffe-e4b5-4284-b1eb-a955b894735e'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:38.776625 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbde16-268b-4084-aefd-55ac7a10fd14 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:38.830452 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f3cffffe-e4b5-4284-b1eb-a955b894735e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f93d2a4f8f4f4e888f3949bba39b6e4e', 'request_id': 'req-f3cffffe-e4b5-4284-b1eb-a955b894735e'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:38.839131 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f3cffffe-e4b5-4284-b1eb-a955b894735e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f93d2a4f8f4f4e888f3949bba39b6e4e', 'request_id': 'req-f3cffffe-e4b5-4284-b1eb-a955b894735e'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:38.841014 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba29c6-58ab-412e-b572-40857909de95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:38.841730 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba29c6-58ab-412e-b572-40857909de95 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:38.842023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba29c6-58ab-412e-b572-40857909de95 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:38.842290 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba29c6-58ab-412e-b572-40857909de95 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:38.843009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba29c6-58ab-412e-b572-40857909de95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:38.844657 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba29c6-58ab-412e-b572-40857909de95 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:38.850397 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f3cffffe-e4b5-4284-b1eb-a955b894735e admin admin] The token cache is being invalidate because project f93d2a4f8f4f4e888f3949bba39b6e4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:54:38.850689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f3cffffe-e4b5-4284-b1eb-a955b894735e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:38.851359 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f3cffffe-e4b5-4284-b1eb-a955b894735e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:38.853653 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 946/1912] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:54:38 2022] DELETE /identity/v3/projects/f93d2a4f8f4f4e888f3949bba39b6e4e => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:38.857311 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cffffe-e4b5-4284-b1eb-a955b894735e admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:38.862662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 967/1913] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:38.862786 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-f3cffffe-e4b5-4284-b1eb-a955b894735e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:38.866698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba29c6-58ab-412e-b572-40857909de95 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:38.868417 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-31ba29c6-58ab-412e-b572-40857909de95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:38.880190 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cffffe-e4b5-4284-b1eb-a955b894735e admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:38.882714 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba29c6-58ab-412e-b572-40857909de95 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:38.885124 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e9846-bbcd-4374-a585-45f5aba57b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:38.886042 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e9846-bbcd-4374-a585-45f5aba57b15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:38.886385 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e9846-bbcd-4374-a585-45f5aba57b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:38.887078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e9846-bbcd-4374-a585-45f5aba57b15 admin admin] PATH_INFO: `/v3/users/1e188c777ed341168e7ab8ba15c972e7` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:38.887829 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c425f-89da-4039-bbbb-8c5154b504bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:38.887954 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e9846-bbcd-4374-a585-45f5aba57b15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e188c777ed341168e7ab8ba15c972e7)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:38.888651 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c425f-89da-4039-bbbb-8c5154b504bf admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:38.890448 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c425f-89da-4039-bbbb-8c5154b504bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:38.890448 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c425f-89da-4039-bbbb-8c5154b504bf admin admin] PATH_INFO: `/v3/projects/4c5b35039a1846b8862f2f17449f18b9/users/85778cd9234949fb8a8cecb9b01320a5/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:38.890617 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e9846-bbcd-4374-a585-45f5aba57b15 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:38.891596 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c425f-89da-4039-bbbb-8c5154b504bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c5b35039a1846b8862f2f17449f18b9, user_id=85778cd9234949fb8a8cecb9b01320a5, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:38.894289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c425f-89da-4039-bbbb-8c5154b504bf admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:38.906209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 968/1914] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:38 2022] PUT /identity/v3/projects/4c5b35039a1846b8862f2f17449f18b9/users/85778cd9234949fb8a8cecb9b01320a5/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:39.155839 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-8e4e9846-bbcd-4374-a585-45f5aba57b15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e188c777ed341168e7ab8ba15c972e7'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:39.169930 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-8e4e9846-bbcd-4374-a585-45f5aba57b15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e188c777ed341168e7ab8ba15c972e7'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:39.288251 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-8e4e9846-bbcd-4374-a585-45f5aba57b15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e188c777ed341168e7ab8ba15c972e7'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:39.290464 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-8e4e9846-bbcd-4374-a585-45f5aba57b15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e188c777ed341168e7ab8ba15c972e7'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:39.291177 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-8e4e9846-bbcd-4374-a585-45f5aba57b15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e188c777ed341168e7ab8ba15c972e7'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:39.309563 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 947/1915] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:54:38 2022] DELETE /identity/v3/users/1e188c777ed341168e7ab8ba15c972e7 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:54:39.719083 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c425f-89da-4039-bbbb-8c5154b504bf admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:39.721736 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-4c5c425f-89da-4039-bbbb-8c5154b504bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:39.770757 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:39.794755 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c425f-89da-4039-bbbb-8c5154b504bf admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:39.871225 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f5f42-047b-4d20-a051-80565fc2e2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:39.871907 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f5f42-047b-4d20-a051-80565fc2e2cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:39.872193 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f5f42-047b-4d20-a051-80565fc2e2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:39.872492 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f5f42-047b-4d20-a051-80565fc2e2cb admin admin] PATH_INFO: `/v3/projects/c3de64cde16d464c8f4b1775c82fc4bd` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:39.873662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f5f42-047b-4d20-a051-80565fc2e2cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3de64cde16d464c8f4b1775c82fc4bd)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:39.875736 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f5f42-047b-4d20-a051-80565fc2e2cb admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:39.974475 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f67f5f42-047b-4d20-a051-80565fc2e2cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3de64cde16d464c8f4b1775c82fc4bd', 'request_id': 'req-f67f5f42-047b-4d20-a051-80565fc2e2cb'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:40.024656 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f67f5f42-047b-4d20-a051-80565fc2e2cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3de64cde16d464c8f4b1775c82fc4bd', 'request_id': 'req-f67f5f42-047b-4d20-a051-80565fc2e2cb'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:40.030998 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f67f5f42-047b-4d20-a051-80565fc2e2cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3de64cde16d464c8f4b1775c82fc4bd', 'request_id': 'req-f67f5f42-047b-4d20-a051-80565fc2e2cb'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:40.044258 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f67f5f42-047b-4d20-a051-80565fc2e2cb admin admin] The token cache is being invalidate because project c3de64cde16d464c8f4b1775c82fc4bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:54:40.044744 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f67f5f42-047b-4d20-a051-80565fc2e2cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:40.045453 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f67f5f42-047b-4d20-a051-80565fc2e2cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:40.047415 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 969/1916] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:54:39 2022] DELETE /identity/v3/projects/c3de64cde16d464c8f4b1775c82fc4bd => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:40.062709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831706b-3bfb-46e3-81a8-54a320f98ae1 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:40.063141 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831706b-3bfb-46e3-81a8-54a320f98ae1 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:40.063432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831706b-3bfb-46e3-81a8-54a320f98ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:40.084103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-6831706b-3bfb-46e3-81a8-54a320f98ae1 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:40.085256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-6831706b-3bfb-46e3-81a8-54a320f98ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:40.137024 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:40.150070 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 948/1917] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:40.157941 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8121f-165f-4dc0-8ca2-2ca11c10dc3d None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:40.158232 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8121f-165f-4dc0-8ca2-2ca11c10dc3d None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:40.158506 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8121f-165f-4dc0-8ca2-2ca11c10dc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:40.180391 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-ccb8121f-165f-4dc0-8ca2-2ca11c10dc3d None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:40.181680 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-ccb8121f-165f-4dc0-8ca2-2ca11c10dc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:40.196820 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831706b-3bfb-46e3-81a8-54a320f98ae1 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:40.202387 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 970/1918] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:40.214864 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831706b-3bfb-46e3-81a8-54a320f98ae1 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:40.225740 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344007ca-f59f-46e2-8dea-c36d1860cf46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:40.226506 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344007ca-f59f-46e2-8dea-c36d1860cf46 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:40.227025 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344007ca-f59f-46e2-8dea-c36d1860cf46 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:40.227377 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344007ca-f59f-46e2-8dea-c36d1860cf46 admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:40.228158 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344007ca-f59f-46e2-8dea-c36d1860cf46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:54:40.228630 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344007ca-f59f-46e2-8dea-c36d1860cf46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:40.230588 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344007ca-f59f-46e2-8dea-c36d1860cf46 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:40.248696 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 949/1919] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:54:40.254865 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb8121f-165f-4dc0-8ca2-2ca11c10dc3d None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:40.267524 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb8121f-165f-4dc0-8ca2-2ca11c10dc3d None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:40.278784 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef9267-a19d-468a-9d8a-5ca9abb0f2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:40.279433 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef9267-a19d-468a-9d8a-5ca9abb0f2d6 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:40.279713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef9267-a19d-468a-9d8a-5ca9abb0f2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:40.279987 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef9267-a19d-468a-9d8a-5ca9abb0f2d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:40.280658 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef9267-a19d-468a-9d8a-5ca9abb0f2d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:40.284357 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef9267-a19d-468a-9d8a-5ca9abb0f2d6 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:40.325327 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f1ebf-3f18-45ed-873a-4ccef9372d25 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:40.325851 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f1ebf-3f18-45ed-873a-4ccef9372d25 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:40.325851 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f1ebf-3f18-45ed-873a-4ccef9372d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:40.359648 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-253f1ebf-3f18-45ed-873a-4ccef9372d25 None None] MFA Rules not processed for user `85778cd9234949fb8a8cecb9b01320a5`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:40.360976 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-253f1ebf-3f18-45ed-873a-4ccef9372d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:40.420579 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 971/1920] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:54:40 2022] POST /identity/v3/projects => generated 367 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:40.422400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef9267-a19d-468a-9d8a-5ca9abb0f2d6 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:40.468814 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 950/1921] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:54:40.481196 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f1ebf-3f18-45ed-873a-4ccef9372d25 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:40.482609 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-253f1ebf-3f18-45ed-873a-4ccef9372d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:40.501555 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:40.517098 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef9267-a19d-468a-9d8a-5ca9abb0f2d6 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:40.522727 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77789e9a-36ef-46f4-b08b-bc0188cf63cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:40.523352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77789e9a-36ef-46f4-b08b-bc0188cf63cd admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:40.523636 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77789e9a-36ef-46f4-b08b-bc0188cf63cd admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:40.523908 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77789e9a-36ef-46f4-b08b-bc0188cf63cd admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:40.524615 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77789e9a-36ef-46f4-b08b-bc0188cf63cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:40.526552 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77789e9a-36ef-46f4-b08b-bc0188cf63cd admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:40.548735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f1ebf-3f18-45ed-873a-4ccef9372d25 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:40.597347 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ddc45-2dc1-4b2f-9b6f-7a3b0c38c401 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:40.598026 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ddc45-2dc1-4b2f-9b6f-7a3b0c38c401 service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:40.598284 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ddc45-2dc1-4b2f-9b6f-7a3b0c38c401 service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:40.598555 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ddc45-2dc1-4b2f-9b6f-7a3b0c38c401 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:40.600420 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-430ddc45-2dc1-4b2f-9b6f-7a3b0c38c401 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:40.659281 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 972/1922] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:54:40 2022] POST /identity/v3/users => generated 284 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:40.663222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ddc45-2dc1-4b2f-9b6f-7a3b0c38c401 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:40.664754 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ddc45-2dc1-4b2f-9b6f-7a3b0c38c401 service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:40.670462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77789e9a-36ef-46f4-b08b-bc0188cf63cd admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:40.674172 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 951/1923] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:54:40.680295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77789e9a-36ef-46f4-b08b-bc0188cf63cd admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:40.685159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ffdcf-796d-4d3b-875c-d498d0bdc3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:40.685801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ffdcf-796d-4d3b-875c-d498d0bdc3a7 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:40.686079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ffdcf-796d-4d3b-875c-d498d0bdc3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:40.686350 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ffdcf-796d-4d3b-875c-d498d0bdc3a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:40.686993 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593ffdcf-796d-4d3b-875c-d498d0bdc3a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:40.688736 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593ffdcf-796d-4d3b-875c-d498d0bdc3a7 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:40.699308 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 973/1924] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 00:54:40.703507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ddc45-2dc1-4b2f-9b6f-7a3b0c38c401 service cinder] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:40.715575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ddc45-2dc1-4b2f-9b6f-7a3b0c38c401 service cinder] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:40.720584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0caf89-0f5c-46f1-8dba-67f793ebfe03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:40.721198 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0caf89-0f5c-46f1-8dba-67f793ebfe03 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:40.721488 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0caf89-0f5c-46f1-8dba-67f793ebfe03 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:40.721760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0caf89-0f5c-46f1-8dba-67f793ebfe03 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:40.722401 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0caf89-0f5c-46f1-8dba-67f793ebfe03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:40.724042 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0caf89-0f5c-46f1-8dba-67f793ebfe03 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:40.734088 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 952/1925] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:40.738391 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ffdcf-796d-4d3b-875c-d498d0bdc3a7 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:40.745024 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:40.748872 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ffdcf-796d-4d3b-875c-d498d0bdc3a7 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:40.754694 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829baa6b-35c5-437a-8a58-4691df9f7662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:40.755375 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829baa6b-35c5-437a-8a58-4691df9f7662 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:40.755660 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829baa6b-35c5-437a-8a58-4691df9f7662 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:40.755940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829baa6b-35c5-437a-8a58-4691df9f7662 admin admin] PATH_INFO: `/v3/projects/e2edd3056a514f4f855a9aa6e78e07ae/users/b1c30e82d09e4abf9e034f9f16db2bf4/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:40.778878 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829baa6b-35c5-437a-8a58-4691df9f7662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2edd3056a514f4f855a9aa6e78e07ae, user_id=b1c30e82d09e4abf9e034f9f16db2bf4, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:40.783832 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829baa6b-35c5-437a-8a58-4691df9f7662 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:40.887632 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 974/1926] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:40 2022] PUT /identity/v3/projects/e2edd3056a514f4f855a9aa6e78e07ae/users/b1c30e82d09e4abf9e034f9f16db2bf4/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:40.903540 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b57ac5-5ffd-45be-ac06-91bd1f9abeaa None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:40.904351 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b57ac5-5ffd-45be-ac06-91bd1f9abeaa None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:40.904885 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b57ac5-5ffd-45be-ac06-91bd1f9abeaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:40.934038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-26b57ac5-5ffd-45be-ac06-91bd1f9abeaa None None] MFA Rules not processed for user `b1c30e82d09e4abf9e034f9f16db2bf4`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:40.935313 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-26b57ac5-5ffd-45be-ac06-91bd1f9abeaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:40.999790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 953/1927] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:41.004385 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829baa6b-35c5-437a-8a58-4691df9f7662 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:41.065827 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829baa6b-35c5-437a-8a58-4691df9f7662 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:41.071057 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703b1d4-47b4-4dc0-8c2b-ae052b55e82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:41.071666 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703b1d4-47b4-4dc0-8c2b-ae052b55e82a admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:41.072040 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703b1d4-47b4-4dc0-8c2b-ae052b55e82a admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:41.072220 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703b1d4-47b4-4dc0-8c2b-ae052b55e82a admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:41.072918 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703b1d4-47b4-4dc0-8c2b-ae052b55e82a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:41.074858 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703b1d4-47b4-4dc0-8c2b-ae052b55e82a admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:41.186967 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 975/1928] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:54:41 2022] POST /identity/v3/projects => generated 365 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:41.191238 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b57ac5-5ffd-45be-ac06-91bd1f9abeaa None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:41.203014 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b57ac5-5ffd-45be-ac06-91bd1f9abeaa None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:41.265760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cc958-f74a-4459-920c-1ed3ae389107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:41.266443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cc958-f74a-4459-920c-1ed3ae389107 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:41.266732 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cc958-f74a-4459-920c-1ed3ae389107 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:41.267006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cc958-f74a-4459-920c-1ed3ae389107 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:41.267647 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904cc958-f74a-4459-920c-1ed3ae389107 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:41.269670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904cc958-f74a-4459-920c-1ed3ae389107 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:41.349088 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 954/1929] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:54:41 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:41.350358 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703b1d4-47b4-4dc0-8c2b-ae052b55e82a admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:41.361381 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703b1d4-47b4-4dc0-8c2b-ae052b55e82a admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:41.367417 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4adad-6846-441c-be0f-edeffb63ff5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:41.369650 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4adad-6846-441c-be0f-edeffb63ff5d admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:41.370065 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4adad-6846-441c-be0f-edeffb63ff5d admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:41.370405 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4adad-6846-441c-be0f-edeffb63ff5d admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:41.371125 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c4adad-6846-441c-be0f-edeffb63ff5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:41.372820 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c4adad-6846-441c-be0f-edeffb63ff5d admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:41.387842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 976/1930] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:41.388977 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cc958-f74a-4459-920c-1ed3ae389107 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:41.405657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cc958-f74a-4459-920c-1ed3ae389107 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:41.416083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c00cd4-4ce2-4993-9c1d-182b10c9462e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:41.418622 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c00cd4-4ce2-4993-9c1d-182b10c9462e admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:41.418969 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c00cd4-4ce2-4993-9c1d-182b10c9462e admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:41.419206 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c00cd4-4ce2-4993-9c1d-182b10c9462e admin admin] PATH_INFO: `/v3/projects/b606d11bf12d45d79ea2e2c8308cd92d/users/0633a8a908754a958b88af32f34f0d61/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:41.445598 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c00cd4-4ce2-4993-9c1d-182b10c9462e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b606d11bf12d45d79ea2e2c8308cd92d, user_id=0633a8a908754a958b88af32f34f0d61, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:41.449739 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c00cd4-4ce2-4993-9c1d-182b10c9462e admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:41.527913 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 955/1931] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:41 2022] PUT /identity/v3/projects/b606d11bf12d45d79ea2e2c8308cd92d/users/0633a8a908754a958b88af32f34f0d61/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:41.532116 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4adad-6846-441c-be0f-edeffb63ff5d admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:41.550069 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4adad-6846-441c-be0f-edeffb63ff5d admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:41.616038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c6b2b-b9c3-4e25-81d0-eb0762c63a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:41.616692 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c6b2b-b9c3-4e25-81d0-eb0762c63a99 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:41.618249 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c6b2b-b9c3-4e25-81d0-eb0762c63a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:41.618249 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c6b2b-b9c3-4e25-81d0-eb0762c63a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:41.618249 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c6b2b-b9c3-4e25-81d0-eb0762c63a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:41.619491 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c6b2b-b9c3-4e25-81d0-eb0762c63a99 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:41.630032 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 977/1932] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 00:54:41.634268 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c00cd4-4ce2-4993-9c1d-182b10c9462e admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:41.644410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c00cd4-4ce2-4993-9c1d-182b10c9462e admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:41.649170 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67582f-92c5-48df-91a2-d3b5894c4630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:41.652627 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67582f-92c5-48df-91a2-d3b5894c4630 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:41.652627 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67582f-92c5-48df-91a2-d3b5894c4630 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:41.652627 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67582f-92c5-48df-91a2-d3b5894c4630 admin admin] PATH_INFO: `/v3/projects/b606d11bf12d45d79ea2e2c8308cd92d/users/0633a8a908754a958b88af32f34f0d61/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:41.652627 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c67582f-92c5-48df-91a2-d3b5894c4630 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b606d11bf12d45d79ea2e2c8308cd92d, user_id=0633a8a908754a958b88af32f34f0d61, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:41.654833 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c67582f-92c5-48df-91a2-d3b5894c4630 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:41.665791 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 956/1933] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:41 2022] PUT /identity/v3/projects/b606d11bf12d45d79ea2e2c8308cd92d/users/0633a8a908754a958b88af32f34f0d61/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:41.676853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894682c1-2cd0-4e3e-ac22-cc1ac2207efe None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:41.677160 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894682c1-2cd0-4e3e-ac22-cc1ac2207efe None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:41.677413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894682c1-2cd0-4e3e-ac22-cc1ac2207efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:41.699280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:41.706814 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-894682c1-2cd0-4e3e-ac22-cc1ac2207efe None None] MFA Rules not processed for user `0633a8a908754a958b88af32f34f0d61`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:41.708159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-894682c1-2cd0-4e3e-ac22-cc1ac2207efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:41.778383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 978/1934] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:41 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:41.790197 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67582f-92c5-48df-91a2-d3b5894c4630 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:41.799629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:41.847765 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67582f-92c5-48df-91a2-d3b5894c4630 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:41.851441 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02723b-fb2d-4a4d-8e90-b4991fa892cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:41.852097 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02723b-fb2d-4a4d-8e90-b4991fa892cf service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:41.852378 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02723b-fb2d-4a4d-8e90-b4991fa892cf service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:41.852685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02723b-fb2d-4a4d-8e90-b4991fa892cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:41.862059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc02723b-fb2d-4a4d-8e90-b4991fa892cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:41.864024 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc02723b-fb2d-4a4d-8e90-b4991fa892cf service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:41.873781 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 957/1935] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:41 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:54:42.072259 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894682c1-2cd0-4e3e-ac22-cc1ac2207efe None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:42.081590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894682c1-2cd0-4e3e-ac22-cc1ac2207efe None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:42.085316 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad2627e-d62e-47b4-9c11-44c7a31b01cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:42.086150 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2627e-d62e-47b4-9c11-44c7a31b01cb service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:42.086447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2627e-d62e-47b4-9c11-44c7a31b01cb service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:42.086726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2627e-d62e-47b4-9c11-44c7a31b01cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:42.143213 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad2627e-d62e-47b4-9c11-44c7a31b01cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:42.144716 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad2627e-d62e-47b4-9c11-44c7a31b01cb service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:42.153740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 979/1936] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:54:45.674512 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02723b-fb2d-4a4d-8e90-b4991fa892cf service cinder] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:45.683808 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02723b-fb2d-4a4d-8e90-b4991fa892cf service cinder] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:45.687877 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0edd8-9994-49b8-b7c4-715ab3d923b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:45.689079 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0edd8-9994-49b8-b7c4-715ab3d923b7 service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:45.689194 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0edd8-9994-49b8-b7c4-715ab3d923b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:45.689475 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0edd8-9994-49b8-b7c4-715ab3d923b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:45.749982 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0edd8-9994-49b8-b7c4-715ab3d923b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:45.751426 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0edd8-9994-49b8-b7c4-715ab3d923b7 service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:45.762080 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 958/1937] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:54:50.807415 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad2627e-d62e-47b4-9c11-44c7a31b01cb service cinder] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:50.810312 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-5ad2627e-d62e-47b4-9c11-44c7a31b01cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:50.909619 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad2627e-d62e-47b4-9c11-44c7a31b01cb service cinder] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:50.916436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d93b1-768f-4e6c-9fd9-de1ced61059c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:50.917843 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d93b1-768f-4e6c-9fd9-de1ced61059c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:50.917843 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d93b1-768f-4e6c-9fd9-de1ced61059c admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:50.917843 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d93b1-768f-4e6c-9fd9-de1ced61059c admin admin] PATH_INFO: `/v3/users/e68a4a654a1e4a49b9e59cff12b68e72` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:50.919539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d93b1-768f-4e6c-9fd9-de1ced61059c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e68a4a654a1e4a49b9e59cff12b68e72)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:50.922852 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d93b1-768f-4e6c-9fd9-de1ced61059c admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:51.126046 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:51.142316 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-b98d93b1-768f-4e6c-9fd9-de1ced61059c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e68a4a654a1e4a49b9e59cff12b68e72'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:51.143378 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-b98d93b1-768f-4e6c-9fd9-de1ced61059c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e68a4a654a1e4a49b9e59cff12b68e72'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:51.166612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-b98d93b1-768f-4e6c-9fd9-de1ced61059c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e68a4a654a1e4a49b9e59cff12b68e72'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:51.287712 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-b98d93b1-768f-4e6c-9fd9-de1ced61059c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e68a4a654a1e4a49b9e59cff12b68e72'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:51.288436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-b98d93b1-768f-4e6c-9fd9-de1ced61059c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e68a4a654a1e4a49b9e59cff12b68e72'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:51.307868 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 980/1938] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:54:50 2022] DELETE /identity/v3/users/e68a4a654a1e4a49b9e59cff12b68e72 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:51.313360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb513d-b62a-4103-805e-3b8f5205bcda None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:51.315807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb513d-b62a-4103-805e-3b8f5205bcda None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:51.317593 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb513d-b62a-4103-805e-3b8f5205bcda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:51.341573 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-1fbb513d-b62a-4103-805e-3b8f5205bcda None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:51.343373 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-1fbb513d-b62a-4103-805e-3b8f5205bcda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:51.432578 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d93b1-768f-4e6c-9fd9-de1ced61059c admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:51.439322 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 959/1939] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:54:51.447856 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-b98d93b1-768f-4e6c-9fd9-de1ced61059c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:51.461152 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb513d-b62a-4103-805e-3b8f5205bcda None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:51.462959 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-1fbb513d-b62a-4103-805e-3b8f5205bcda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:51.474450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d93b1-768f-4e6c-9fd9-de1ced61059c admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:51.476449 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ffd02-fdbe-44e1-850c-042f074b199e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:51.477165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ffd02-fdbe-44e1-850c-042f074b199e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:51.477447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ffd02-fdbe-44e1-850c-042f074b199e admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:51.477779 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ffd02-fdbe-44e1-850c-042f074b199e admin admin] PATH_INFO: `/v3/users/b1c30e82d09e4abf9e034f9f16db2bf4` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:51.479178 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ffd02-fdbe-44e1-850c-042f074b199e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1c30e82d09e4abf9e034f9f16db2bf4)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:51.482057 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ffd02-fdbe-44e1-850c-042f074b199e admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:51.549178 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:51.549178 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb513d-b62a-4103-805e-3b8f5205bcda None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:51.549178 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f71a6-6781-4f49-b941-8b22e404262e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:51.549178 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f71a6-6781-4f49-b941-8b22e404262e service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:51.550215 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f71a6-6781-4f49-b941-8b22e404262e service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:51.550215 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f71a6-6781-4f49-b941-8b22e404262e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:51.559510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f71a6-6781-4f49-b941-8b22e404262e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:51.561151 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f71a6-6781-4f49-b941-8b22e404262e service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:51.572004 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 960/1940] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:54:52.025796 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ca4ffd02-fdbe-44e1-850c-042f074b199e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1c30e82d09e4abf9e034f9f16db2bf4'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:52.025796 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ca4ffd02-fdbe-44e1-850c-042f074b199e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1c30e82d09e4abf9e034f9f16db2bf4'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:52.053312 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ca4ffd02-fdbe-44e1-850c-042f074b199e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1c30e82d09e4abf9e034f9f16db2bf4'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:52.110222 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ca4ffd02-fdbe-44e1-850c-042f074b199e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1c30e82d09e4abf9e034f9f16db2bf4'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:52.110926 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ca4ffd02-fdbe-44e1-850c-042f074b199e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1c30e82d09e4abf9e034f9f16db2bf4'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:52.126884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 981/1941] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:54:51 2022] DELETE /identity/v3/users/b1c30e82d09e4abf9e034f9f16db2bf4 => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:52.132735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e96013-5a4f-4ee4-965a-88d5abfceafe None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:52.133026 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e96013-5a4f-4ee4-965a-88d5abfceafe None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:52.133303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e96013-5a4f-4ee4-965a-88d5abfceafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:52.160015 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-b4e96013-5a4f-4ee4-965a-88d5abfceafe None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:52.164462 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-b4e96013-5a4f-4ee4-965a-88d5abfceafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:52.250755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 961/1942] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:52.263891 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ffd02-fdbe-44e1-850c-042f074b199e admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:52.270263 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-ca4ffd02-fdbe-44e1-850c-042f074b199e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:52.387831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ffd02-fdbe-44e1-850c-042f074b199e admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:52.391773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c628d85-d7c1-4800-8a2b-5574ab2fabdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:52.392430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c628d85-d7c1-4800-8a2b-5574ab2fabdc service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:52.392734 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c628d85-d7c1-4800-8a2b-5574ab2fabdc service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:52.393003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c628d85-d7c1-4800-8a2b-5574ab2fabdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:52.412271 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c628d85-d7c1-4800-8a2b-5574ab2fabdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:52.414194 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c628d85-d7c1-4800-8a2b-5574ab2fabdc service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:52.426822 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 982/1943] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:54:52.470066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e96013-5a4f-4ee4-965a-88d5abfceafe None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:52.471584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-b4e96013-5a4f-4ee4-965a-88d5abfceafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:52.489603 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e96013-5a4f-4ee4-965a-88d5abfceafe None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:52.496915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106947e6-ed02-4269-98b4-184515d79d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:52.497652 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106947e6-ed02-4269-98b4-184515d79d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:52.509063 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106947e6-ed02-4269-98b4-184515d79d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:52.509358 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106947e6-ed02-4269-98b4-184515d79d40 admin admin] PATH_INFO: `/v3/projects/ce7591f88899478b9bc71cd16ef37897` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:52.514148 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106947e6-ed02-4269-98b4-184515d79d40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce7591f88899478b9bc71cd16ef37897)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:52.518483 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106947e6-ed02-4269-98b4-184515d79d40 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:52.799510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-106947e6-ed02-4269-98b4-184515d79d40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce7591f88899478b9bc71cd16ef37897', 'request_id': 'req-106947e6-ed02-4269-98b4-184515d79d40'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:52.899389 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-106947e6-ed02-4269-98b4-184515d79d40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce7591f88899478b9bc71cd16ef37897', 'request_id': 'req-106947e6-ed02-4269-98b4-184515d79d40'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:52.914784 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-106947e6-ed02-4269-98b4-184515d79d40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce7591f88899478b9bc71cd16ef37897', 'request_id': 'req-106947e6-ed02-4269-98b4-184515d79d40'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:52.927395 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-106947e6-ed02-4269-98b4-184515d79d40 admin admin] The token cache is being invalidate because project ce7591f88899478b9bc71cd16ef37897 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:54:52.927916 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-106947e6-ed02-4269-98b4-184515d79d40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:52.928740 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-106947e6-ed02-4269-98b4-184515d79d40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:52.930538 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 962/1944] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:54:52 2022] DELETE /identity/v3/projects/ce7591f88899478b9bc71cd16ef37897 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:52.936949 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c628d85-d7c1-4800-8a2b-5574ab2fabdc service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:52.949208 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c628d85-d7c1-4800-8a2b-5574ab2fabdc service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:53.005419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:53.085402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d92ac-1544-4d76-ac27-92924d55a64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:53.087763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d92ac-1544-4d76-ac27-92924d55a64e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:53.088060 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d92ac-1544-4d76-ac27-92924d55a64e admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:53.088351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d92ac-1544-4d76-ac27-92924d55a64e admin admin] PATH_INFO: `/v3/users/032ed13033534f178a06f6b361915f29` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:53.089630 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052d92ac-1544-4d76-ac27-92924d55a64e admin admin] RBAC: Authorizing `identity:delete_user(user_id=032ed13033534f178a06f6b361915f29)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:53.092807 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052d92ac-1544-4d76-ac27-92924d55a64e admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:53.280660 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-052d92ac-1544-4d76-ac27-92924d55a64e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '032ed13033534f178a06f6b361915f29'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:53.281355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-052d92ac-1544-4d76-ac27-92924d55a64e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '032ed13033534f178a06f6b361915f29'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:53.296924 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-052d92ac-1544-4d76-ac27-92924d55a64e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '032ed13033534f178a06f6b361915f29'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:53.355961 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-052d92ac-1544-4d76-ac27-92924d55a64e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '032ed13033534f178a06f6b361915f29'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:53.356761 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-052d92ac-1544-4d76-ac27-92924d55a64e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '032ed13033534f178a06f6b361915f29'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:53.372698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 983/1945] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:54:52 2022] DELETE /identity/v3/users/032ed13033534f178a06f6b361915f29 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:53.467427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106947e6-ed02-4269-98b4-184515d79d40 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:53.469085 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-106947e6-ed02-4269-98b4-184515d79d40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:53.493822 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106947e6-ed02-4269-98b4-184515d79d40 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:53.589058 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e1139-2931-499c-970b-6e023ad2edd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:53.589958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e1139-2931-499c-970b-6e023ad2edd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:53.590245 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e1139-2931-499c-970b-6e023ad2edd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:53.591416 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e1139-2931-499c-970b-6e023ad2edd4 admin admin] PATH_INFO: `/v3/projects/e2edd3056a514f4f855a9aa6e78e07ae` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:53.593281 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e1139-2931-499c-970b-6e023ad2edd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2edd3056a514f4f855a9aa6e78e07ae)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:53.596874 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e1139-2931-499c-970b-6e023ad2edd4 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:53.613165 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:53.801643 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-185e1139-2931-499c-970b-6e023ad2edd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2edd3056a514f4f855a9aa6e78e07ae', 'request_id': 'req-185e1139-2931-499c-970b-6e023ad2edd4'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:53.910852 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-185e1139-2931-499c-970b-6e023ad2edd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2edd3056a514f4f855a9aa6e78e07ae', 'request_id': 'req-185e1139-2931-499c-970b-6e023ad2edd4'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:53.918173 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-185e1139-2931-499c-970b-6e023ad2edd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2edd3056a514f4f855a9aa6e78e07ae', 'request_id': 'req-185e1139-2931-499c-970b-6e023ad2edd4'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:53.935641 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-185e1139-2931-499c-970b-6e023ad2edd4 admin admin] The token cache is being invalidate because project e2edd3056a514f4f855a9aa6e78e07ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:54:53.935641 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-185e1139-2931-499c-970b-6e023ad2edd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:53.937243 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-185e1139-2931-499c-970b-6e023ad2edd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:53.939105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 963/1946] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:54:53 2022] DELETE /identity/v3/projects/e2edd3056a514f4f855a9aa6e78e07ae => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:53.942945 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d92ac-1544-4d76-ac27-92924d55a64e admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:53.944459 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-052d92ac-1544-4d76-ac27-92924d55a64e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:53.959324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d92ac-1544-4d76-ac27-92924d55a64e admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:54.024365 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e1139-2931-499c-970b-6e023ad2edd4 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:54.026046 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-185e1139-2931-499c-970b-6e023ad2edd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:54.033790 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddafdc1-18fd-48cd-8b73-2d498e3aac4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:54.036962 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddafdc1-18fd-48cd-8b73-2d498e3aac4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:54.038031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddafdc1-18fd-48cd-8b73-2d498e3aac4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:54.039089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddafdc1-18fd-48cd-8b73-2d498e3aac4e admin admin] PATH_INFO: `/v3/users/0633a8a908754a958b88af32f34f0d61` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:54.040179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddafdc1-18fd-48cd-8b73-2d498e3aac4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0633a8a908754a958b88af32f34f0d61)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:54.043409 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddafdc1-18fd-48cd-8b73-2d498e3aac4e admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:54.136254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e1139-2931-499c-970b-6e023ad2edd4 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:54.138729 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc3aa3-4c79-4590-8b4f-97bfe4729d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:54.139375 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc3aa3-4c79-4590-8b4f-97bfe4729d4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:54.139653 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc3aa3-4c79-4590-8b4f-97bfe4729d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:54.139952 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc3aa3-4c79-4590-8b4f-97bfe4729d4d admin admin] PATH_INFO: `/v3/projects/ebf050447dd84b188ae25db1fc470aa0` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:54.141275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc3aa3-4c79-4590-8b4f-97bfe4729d4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebf050447dd84b188ae25db1fc470aa0)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:54.143482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc3aa3-4c79-4590-8b4f-97bfe4729d4d admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:54.251233 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-6ddafdc1-18fd-48cd-8b73-2d498e3aac4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0633a8a908754a958b88af32f34f0d61'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:54.252024 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-6ddafdc1-18fd-48cd-8b73-2d498e3aac4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0633a8a908754a958b88af32f34f0d61'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:54.254258 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:54.257573 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f7dc3aa3-4c79-4590-8b4f-97bfe4729d4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebf050447dd84b188ae25db1fc470aa0', 'request_id': 'req-f7dc3aa3-4c79-4590-8b4f-97bfe4729d4d'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:54.266520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-6ddafdc1-18fd-48cd-8b73-2d498e3aac4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0633a8a908754a958b88af32f34f0d61'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:54.326841 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f7dc3aa3-4c79-4590-8b4f-97bfe4729d4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebf050447dd84b188ae25db1fc470aa0', 'request_id': 'req-f7dc3aa3-4c79-4590-8b4f-97bfe4729d4d'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:54.334285 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f7dc3aa3-4c79-4590-8b4f-97bfe4729d4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebf050447dd84b188ae25db1fc470aa0', 'request_id': 'req-f7dc3aa3-4c79-4590-8b4f-97bfe4729d4d'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:54.345408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f7dc3aa3-4c79-4590-8b4f-97bfe4729d4d admin admin] The token cache is being invalidate because project ebf050447dd84b188ae25db1fc470aa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:54:54.345697 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f7dc3aa3-4c79-4590-8b4f-97bfe4729d4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:54.346372 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f7dc3aa3-4c79-4590-8b4f-97bfe4729d4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:54.348442 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 964/1947] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:54:54 2022] DELETE /identity/v3/projects/ebf050447dd84b188ae25db1fc470aa0 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:54:54.353423 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ad360-9094-47c6-8733-2f39c18fc8be None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:54.353766 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ad360-9094-47c6-8733-2f39c18fc8be None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:54.354051 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ad360-9094-47c6-8733-2f39c18fc8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:54.374698 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-003ad360-9094-47c6-8733-2f39c18fc8be None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:54.375960 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-003ad360-9094-47c6-8733-2f39c18fc8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:54.396486 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-6ddafdc1-18fd-48cd-8b73-2d498e3aac4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0633a8a908754a958b88af32f34f0d61'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:54.399355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-6ddafdc1-18fd-48cd-8b73-2d498e3aac4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0633a8a908754a958b88af32f34f0d61'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:54.403543 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:54.421608 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 984/1948] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:54:53 2022] DELETE /identity/v3/users/0633a8a908754a958b88af32f34f0d61 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:54.452087 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 965/1949] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:54:54.464482 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee202226-386e-479f-88ed-e91188cbfe1a None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:54.464790 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee202226-386e-479f-88ed-e91188cbfe1a None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:54.465058 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee202226-386e-479f-88ed-e91188cbfe1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:54.487695 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-ee202226-386e-479f-88ed-e91188cbfe1a None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:54.489052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-ee202226-386e-479f-88ed-e91188cbfe1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:54.558674 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 985/1950] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:54.563119 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ad360-9094-47c6-8733-2f39c18fc8be None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:54.574401 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ad360-9094-47c6-8733-2f39c18fc8be None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:54.579431 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50ae3b-769f-4186-ad50-b10d4928a47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:54.584595 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50ae3b-769f-4186-ad50-b10d4928a47b admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:54.584595 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50ae3b-769f-4186-ad50-b10d4928a47b admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:54.584595 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50ae3b-769f-4186-ad50-b10d4928a47b admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:54.584595 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50ae3b-769f-4186-ad50-b10d4928a47b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:54:54.584595 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50ae3b-769f-4186-ad50-b10d4928a47b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:54.584595 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50ae3b-769f-4186-ad50-b10d4928a47b admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:54.595769 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 966/1951] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:54:54.600221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee202226-386e-479f-88ed-e91188cbfe1a None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:54.609763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee202226-386e-479f-88ed-e91188cbfe1a None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:54.616012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e64c6-b3c9-4ec6-a4fc-c4a008aeda08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:54.616766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e64c6-b3c9-4ec6-a4fc-c4a008aeda08 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:54.617050 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e64c6-b3c9-4ec6-a4fc-c4a008aeda08 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:54.617323 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e64c6-b3c9-4ec6-a4fc-c4a008aeda08 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:54.618034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e64c6-b3c9-4ec6-a4fc-c4a008aeda08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:54.619979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e64c6-b3c9-4ec6-a4fc-c4a008aeda08 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:54.731958 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 986/1952] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:54:54 2022] POST /identity/v3/projects => generated 357 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:54.736284 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50ae3b-769f-4186-ad50-b10d4928a47b admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:54.747335 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50ae3b-769f-4186-ad50-b10d4928a47b admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:54.812623 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cff02f-4f4f-45e8-8900-2049b29c0a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:54.813399 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cff02f-4f4f-45e8-8900-2049b29c0a02 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:54.813698 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cff02f-4f4f-45e8-8900-2049b29c0a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:54.813979 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cff02f-4f4f-45e8-8900-2049b29c0a02 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:54.814598 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cff02f-4f4f-45e8-8900-2049b29c0a02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:54.817130 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cff02f-4f4f-45e8-8900-2049b29c0a02 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:54.927526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 967/1953] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:54:54 2022] POST /identity/v3/users => generated 279 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:54.931903 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e64c6-b3c9-4ec6-a4fc-c4a008aeda08 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:54.950352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e64c6-b3c9-4ec6-a4fc-c4a008aeda08 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:54.965036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cff02f-4f4f-45e8-8900-2049b29c0a02 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:54.967577 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-61cff02f-4f4f-45e8-8900-2049b29c0a02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:54.968860 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399cd17-3966-4ff0-970b-259c8e023c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:54.969490 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399cd17-3966-4ff0-970b-259c8e023c1d admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:54.969772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399cd17-3966-4ff0-970b-259c8e023c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:54.970043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399cd17-3966-4ff0-970b-259c8e023c1d admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:54.970703 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399cd17-3966-4ff0-970b-259c8e023c1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:54.972804 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399cd17-3966-4ff0-970b-259c8e023c1d admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:54.982843 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cff02f-4f4f-45e8-8900-2049b29c0a02 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:54.991055 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bfe3e0-7f9c-49c8-92fb-f41776a7be7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:54.995151 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfe3e0-7f9c-49c8-92fb-f41776a7be7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:54.995255 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 987/1954] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:54.995338 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfe3e0-7f9c-49c8-92fb-f41776a7be7f admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:54.995338 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfe3e0-7f9c-49c8-92fb-f41776a7be7f admin admin] PATH_INFO: `/v3/projects/b606d11bf12d45d79ea2e2c8308cd92d` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:54.995338 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bfe3e0-7f9c-49c8-92fb-f41776a7be7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b606d11bf12d45d79ea2e2c8308cd92d)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:54.995683 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bfe3e0-7f9c-49c8-92fb-f41776a7be7f admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:54.996125 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399cd17-3966-4ff0-970b-259c8e023c1d admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:55.015959 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399cd17-3966-4ff0-970b-259c8e023c1d admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:55.023253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc47d7-eb72-4f8c-b5ba-f8c7fa90a95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:55.024145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc47d7-eb72-4f8c-b5ba-f8c7fa90a95c admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:55.024632 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc47d7-eb72-4f8c-b5ba-f8c7fa90a95c admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:55.025214 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc47d7-eb72-4f8c-b5ba-f8c7fa90a95c admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:55.026318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cc47d7-eb72-4f8c-b5ba-f8c7fa90a95c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:55.028075 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cc47d7-eb72-4f8c-b5ba-f8c7fa90a95c admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:55.042651 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 988/1955] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:55.046912 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc47d7-eb72-4f8c-b5ba-f8c7fa90a95c admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:55.057559 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc47d7-eb72-4f8c-b5ba-f8c7fa90a95c admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:55.062669 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8307d0f-8295-409b-8232-b64788ddb72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:55.063449 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8307d0f-8295-409b-8232-b64788ddb72b admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:55.063778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8307d0f-8295-409b-8232-b64788ddb72b admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:55.064108 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8307d0f-8295-409b-8232-b64788ddb72b admin admin] PATH_INFO: `/v3/projects/dce7a2fe13bf4130a256418df02eab16/users/e6867be02ee04d22819849a576c13fd5/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:55.079949 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8307d0f-8295-409b-8232-b64788ddb72b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dce7a2fe13bf4130a256418df02eab16, user_id=e6867be02ee04d22819849a576c13fd5, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:55.083268 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8307d0f-8295-409b-8232-b64788ddb72b admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:55.157502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 989/1956] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:55 2022] PUT /identity/v3/projects/dce7a2fe13bf4130a256418df02eab16/users/e6867be02ee04d22819849a576c13fd5/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:54:55.165075 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-b6bfe3e0-7f9c-49c8-92fb-f41776a7be7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b606d11bf12d45d79ea2e2c8308cd92d', 'request_id': 'req-b6bfe3e0-7f9c-49c8-92fb-f41776a7be7f'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:55.168979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa06db9-f938-4231-b97e-1fa861b56b18 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:55.169272 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa06db9-f938-4231-b97e-1fa861b56b18 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:55.169820 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa06db9-f938-4231-b97e-1fa861b56b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:55.208239 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-8fa06db9-f938-4231-b97e-1fa861b56b18 None None] MFA Rules not processed for user `e6867be02ee04d22819849a576c13fd5`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:55.209510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-8fa06db9-f938-4231-b97e-1fa861b56b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:55.258183 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-b6bfe3e0-7f9c-49c8-92fb-f41776a7be7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b606d11bf12d45d79ea2e2c8308cd92d', 'request_id': 'req-b6bfe3e0-7f9c-49c8-92fb-f41776a7be7f'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:55.268658 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-b6bfe3e0-7f9c-49c8-92fb-f41776a7be7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b606d11bf12d45d79ea2e2c8308cd92d', 'request_id': 'req-b6bfe3e0-7f9c-49c8-92fb-f41776a7be7f'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:55.276737 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:55.290839 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-b6bfe3e0-7f9c-49c8-92fb-f41776a7be7f admin admin] The token cache is being invalidate because project b606d11bf12d45d79ea2e2c8308cd92d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:54:55.291137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-b6bfe3e0-7f9c-49c8-92fb-f41776a7be7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:55.295873 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-b6bfe3e0-7f9c-49c8-92fb-f41776a7be7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:54:55.295873 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 968/1957] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:54:54 2022] DELETE /identity/v3/projects/b606d11bf12d45d79ea2e2c8308cd92d => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:54:55.299992 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c856c2-394e-41c9-a12b-f2dad1ac725d None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:55.300279 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c856c2-394e-41c9-a12b-f2dad1ac725d None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:55.304632 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c856c2-394e-41c9-a12b-f2dad1ac725d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:55.313397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 990/1958] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:55 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:54:55.317724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa06db9-f938-4231-b97e-1fa861b56b18 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:55.327803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-b5c856c2-394e-41c9-a12b-f2dad1ac725d None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:55.329333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-b5c856c2-394e-41c9-a12b-f2dad1ac725d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:55.355474 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:55.383858 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa06db9-f938-4231-b97e-1fa861b56b18 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:55.388376 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08634e61-d0e9-4b80-a697-713891d36189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:55.389060 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08634e61-d0e9-4b80-a697-713891d36189 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:55.389345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08634e61-d0e9-4b80-a697-713891d36189 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:55.396794 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08634e61-d0e9-4b80-a697-713891d36189 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:55.396997 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 969/1959] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:55.397114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08634e61-d0e9-4b80-a697-713891d36189 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:55.397114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08634e61-d0e9-4b80-a697-713891d36189 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:55.398596 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e37f3-5326-489a-ace9-a87db4a7a276 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:55.398889 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e37f3-5326-489a-ace9-a87db4a7a276 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:55.399162 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e37f3-5326-489a-ace9-a87db4a7a276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:55.419032 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-f35e37f3-5326-489a-ace9-a87db4a7a276 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:55.420288 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-f35e37f3-5326-489a-ace9-a87db4a7a276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:55.428780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 970/1960] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:55.432973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e37f3-5326-489a-ace9-a87db4a7a276 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:55.443936 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e37f3-5326-489a-ace9-a87db4a7a276 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:55.448746 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f108d3-13c4-4b85-b349-6ae510d67d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:55.449344 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f108d3-13c4-4b85-b349-6ae510d67d90 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:55.449624 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f108d3-13c4-4b85-b349-6ae510d67d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:55.449897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f108d3-13c4-4b85-b349-6ae510d67d90 admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:55.450540 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f108d3-13c4-4b85-b349-6ae510d67d90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:54:55.450841 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f108d3-13c4-4b85-b349-6ae510d67d90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:55.452427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f108d3-13c4-4b85-b349-6ae510d67d90 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:55.464128 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 971/1961] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:54:55.472928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f108d3-13c4-4b85-b349-6ae510d67d90 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:55.473539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 991/1962] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:54:55 2022] POST /identity/v3/projects => generated 357 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:54:55.477232 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08634e61-d0e9-4b80-a697-713891d36189 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:55.479493 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-08634e61-d0e9-4b80-a697-713891d36189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:55.484665 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f108d3-13c4-4b85-b349-6ae510d67d90 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:55.558089 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0f435-7a20-4c4e-82dc-2d62dc0891fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:55.558893 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0f435-7a20-4c4e-82dc-2d62dc0891fe admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:55.559225 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0f435-7a20-4c4e-82dc-2d62dc0891fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:55.559582 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0f435-7a20-4c4e-82dc-2d62dc0891fe admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:55.560344 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0f435-7a20-4c4e-82dc-2d62dc0891fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:55.562848 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0f435-7a20-4c4e-82dc-2d62dc0891fe admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:55.571477 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08634e61-d0e9-4b80-a697-713891d36189 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:55.581918 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46d712-9e14-40b1-9753-ba425642435f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:55.582828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46d712-9e14-40b1-9753-ba425642435f admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:55.583288 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46d712-9e14-40b1-9753-ba425642435f admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:55.583569 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46d712-9e14-40b1-9753-ba425642435f admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:55.584245 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea46d712-9e14-40b1-9753-ba425642435f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:55.586640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea46d712-9e14-40b1-9753-ba425642435f admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:55.722041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 972/1963] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:54:55 2022] POST /identity/v3/projects => generated 385 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:54:55.723748 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0f435-7a20-4c4e-82dc-2d62dc0891fe admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:55.748221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 992/1964] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:54:55 2022] POST /identity/v3/users => generated 285 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:54:55.752019 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46d712-9e14-40b1-9753-ba425642435f admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:55.772652 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46d712-9e14-40b1-9753-ba425642435f admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:55.829937 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0f435-7a20-4c4e-82dc-2d62dc0891fe admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:55.838664 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e1e55c-d068-40ef-879c-913ae329cf47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:55.839992 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e1e55c-d068-40ef-879c-913ae329cf47 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:55.840558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e1e55c-d068-40ef-879c-913ae329cf47 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:55.840871 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e1e55c-d068-40ef-879c-913ae329cf47 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:55.841730 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e1e55c-d068-40ef-879c-913ae329cf47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:55.845148 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e1e55c-d068-40ef-879c-913ae329cf47 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:55.879251 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2e1a0-4d05-4727-a501-9bf07c353c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:55.879893 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2e1a0-4d05-4727-a501-9bf07c353c25 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:55.880199 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2e1a0-4d05-4727-a501-9bf07c353c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:55.880440 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2e1a0-4d05-4727-a501-9bf07c353c25 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:55.881130 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae2e1a0-4d05-4727-a501-9bf07c353c25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:55.882842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae2e1a0-4d05-4727-a501-9bf07c353c25 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:55.891114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:55.896876 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 993/1965] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:55.911573 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2e1a0-4d05-4727-a501-9bf07c353c25 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:55.928272 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2e1a0-4d05-4727-a501-9bf07c353c25 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:55.941642 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 973/1966] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:54:55 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:55.941746 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d78f4-4163-4682-ab2e-e4977041937b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:55.942709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e1e55c-d068-40ef-879c-913ae329cf47 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:55.945539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d78f4-4163-4682-ab2e-e4977041937b admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:55.946044 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d78f4-4163-4682-ab2e-e4977041937b admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:55.946539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d78f4-4163-4682-ab2e-e4977041937b admin admin] PATH_INFO: `/v3/projects/a629d286e21040728b5ec5165dc6bcff/users/abc598f96e2342eb9554055214468335/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:55.975659 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e1e55c-d068-40ef-879c-913ae329cf47 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:55.982197 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c1432-91cb-4bff-bb09-d26f06c9760b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:55.982886 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c1432-91cb-4bff-bb09-d26f06c9760b admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:55.983168 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c1432-91cb-4bff-bb09-d26f06c9760b admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:55.983573 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c1432-91cb-4bff-bb09-d26f06c9760b admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:55.984233 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c1432-91cb-4bff-bb09-d26f06c9760b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:55.986040 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c1432-91cb-4bff-bb09-d26f06c9760b admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:56.005823 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 974/1967] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 00:54:56.007165 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c1432-91cb-4bff-bb09-d26f06c9760b admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:56.008033 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d78f4-4163-4682-ab2e-e4977041937b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a629d286e21040728b5ec5165dc6bcff, user_id=abc598f96e2342eb9554055214468335, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:56.012558 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d78f4-4163-4682-ab2e-e4977041937b admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:56.039127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c1432-91cb-4bff-bb09-d26f06c9760b admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:56.046538 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd8e99-6a80-4dc3-a456-49dedda0bd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:56.047445 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd8e99-6a80-4dc3-a456-49dedda0bd15 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:56.049080 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd8e99-6a80-4dc3-a456-49dedda0bd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:56.049213 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd8e99-6a80-4dc3-a456-49dedda0bd15 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:56.049924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd8e99-6a80-4dc3-a456-49dedda0bd15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:56.052224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd8e99-6a80-4dc3-a456-49dedda0bd15 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:56.088080 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 975/1968] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:56.091475 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd8e99-6a80-4dc3-a456-49dedda0bd15 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:56.101308 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:56.107218 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd8e99-6a80-4dc3-a456-49dedda0bd15 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:56.116726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 994/1969] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:55 2022] PUT /identity/v3/projects/a629d286e21040728b5ec5165dc6bcff/users/abc598f96e2342eb9554055214468335/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:56.120609 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d78f4-4163-4682-ab2e-e4977041937b admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:56.129056 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70b2d0-f38d-4065-a003-156dbf799649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:56.131357 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70b2d0-f38d-4065-a003-156dbf799649 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:56.133161 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70b2d0-f38d-4065-a003-156dbf799649 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:56.145812 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70b2d0-f38d-4065-a003-156dbf799649 admin admin] PATH_INFO: `/v3/projects/a513e6b627b84b8a9e4fa81d1fd8fc4a/users/b7e80549a31b4a4c812f416aca6e2e74/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:56.168732 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd70b2d0-f38d-4065-a003-156dbf799649 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a513e6b627b84b8a9e4fa81d1fd8fc4a, user_id=b7e80549a31b4a4c812f416aca6e2e74, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:56.168732 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd70b2d0-f38d-4065-a003-156dbf799649 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:56.201843 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d78f4-4163-4682-ab2e-e4977041937b admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:56.206456 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ed179-ce91-4f57-b731-e2ab5f0c1250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:56.207102 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ed179-ce91-4f57-b731-e2ab5f0c1250 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:56.207388 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ed179-ce91-4f57-b731-e2ab5f0c1250 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:56.207664 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ed179-ce91-4f57-b731-e2ab5f0c1250 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:56.208308 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46ed179-ce91-4f57-b731-e2ab5f0c1250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:56.209943 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46ed179-ce91-4f57-b731-e2ab5f0c1250 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:56.219774 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 995/1970] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:56.223683 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ed179-ce91-4f57-b731-e2ab5f0c1250 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:56.233231 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ed179-ce91-4f57-b731-e2ab5f0c1250 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:56.238557 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2f19e-419a-4fb8-b5e5-23b79b88c6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:56.239197 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2f19e-419a-4fb8-b5e5-23b79b88c6c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:56.239480 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2f19e-419a-4fb8-b5e5-23b79b88c6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:56.239758 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2f19e-419a-4fb8-b5e5-23b79b88c6c7 admin admin] PATH_INFO: `/v3/projects/a629d286e21040728b5ec5165dc6bcff/users/abc598f96e2342eb9554055214468335/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:56.241682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc2f19e-419a-4fb8-b5e5-23b79b88c6c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a629d286e21040728b5ec5165dc6bcff, user_id=abc598f96e2342eb9554055214468335, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:56.244640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc2f19e-419a-4fb8-b5e5-23b79b88c6c7 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:56.260928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 976/1971] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:56 2022] PUT /identity/v3/projects/a513e6b627b84b8a9e4fa81d1fd8fc4a/users/b7e80549a31b4a4c812f416aca6e2e74/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:54:56.271757 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd7e74-a029-460f-a41b-634b726f7ff9 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:56.272589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd7e74-a029-460f-a41b-634b726f7ff9 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:56.273076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd7e74-a029-460f-a41b-634b726f7ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:56.278671 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 996/1972] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:56 2022] PUT /identity/v3/projects/a629d286e21040728b5ec5165dc6bcff/users/abc598f96e2342eb9554055214468335/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:54:56.292161 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436fb4a-dc93-4f95-bb1d-3ab2e757bc5a None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:56.295637 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436fb4a-dc93-4f95-bb1d-3ab2e757bc5a None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:56.295637 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436fb4a-dc93-4f95-bb1d-3ab2e757bc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:56.326503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-d7bd7e74-a029-460f-a41b-634b726f7ff9 None None] MFA Rules not processed for user `b7e80549a31b4a4c812f416aca6e2e74`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:56.328096 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-d7bd7e74-a029-460f-a41b-634b726f7ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:56.360569 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-1436fb4a-dc93-4f95-bb1d-3ab2e757bc5a None None] MFA Rules not processed for user `abc598f96e2342eb9554055214468335`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:56.361856 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-1436fb4a-dc93-4f95-bb1d-3ab2e757bc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:56.428855 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 977/1973] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:56.445777 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd7e74-a029-460f-a41b-634b726f7ff9 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:56.472545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 997/1974] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:56 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:54:56.520059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd7e74-a029-460f-a41b-634b726f7ff9 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:56.525378 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7da05-8c1f-4488-ae4f-cfaabeba81e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:56.525991 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7da05-8c1f-4488-ae4f-cfaabeba81e0 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:56.526273 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7da05-8c1f-4488-ae4f-cfaabeba81e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:56.526551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7da05-8c1f-4488-ae4f-cfaabeba81e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:56.527180 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7da05-8c1f-4488-ae4f-cfaabeba81e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:56.529186 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7da05-8c1f-4488-ae4f-cfaabeba81e0 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:56.704085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1436fb4a-dc93-4f95-bb1d-3ab2e757bc5a None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:56.705495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-1436fb4a-dc93-4f95-bb1d-3ab2e757bc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:56.757918 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:56.759103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 978/1975] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:54:56 2022] POST /identity/v3/projects => generated 383 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:56.763518 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7da05-8c1f-4488-ae4f-cfaabeba81e0 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:56.776644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1436fb4a-dc93-4f95-bb1d-3ab2e757bc5a None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:56.851318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db334dc-f3ad-4941-9f76-c58a8560862e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:56.851318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db334dc-f3ad-4941-9f76-c58a8560862e service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:56.851646 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db334dc-f3ad-4941-9f76-c58a8560862e service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:56.853558 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db334dc-f3ad-4941-9f76-c58a8560862e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:56.859835 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:56.867303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7da05-8c1f-4488-ae4f-cfaabeba81e0 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:56.873635 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfad2b-8df6-488a-8212-b3d959fa6261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:56.874502 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfad2b-8df6-488a-8212-b3d959fa6261 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:56.874883 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfad2b-8df6-488a-8212-b3d959fa6261 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:56.875275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfad2b-8df6-488a-8212-b3d959fa6261 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:56.876224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfad2b-8df6-488a-8212-b3d959fa6261 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:56.878324 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfad2b-8df6-488a-8212-b3d959fa6261 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:56.923500 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db334dc-f3ad-4941-9f76-c58a8560862e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:56.924983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db334dc-f3ad-4941-9f76-c58a8560862e service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:56.935343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 998/1976] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:56 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:54:56.991158 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 979/1977] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:54:56 2022] POST /identity/v3/users => generated 298 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:54:56.995316 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db334dc-f3ad-4941-9f76-c58a8560862e service cinder] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:57.012265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db334dc-f3ad-4941-9f76-c58a8560862e service cinder] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:57.012265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff60bd-1d08-4579-9bf5-f2640d70480e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:57.012621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff60bd-1d08-4579-9bf5-f2640d70480e admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:57.012906 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff60bd-1d08-4579-9bf5-f2640d70480e admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:57.013182 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff60bd-1d08-4579-9bf5-f2640d70480e admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:57.013828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff60bd-1d08-4579-9bf5-f2640d70480e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:57.015543 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff60bd-1d08-4579-9bf5-f2640d70480e admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:57.025632 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 999/1978] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:57.029875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfad2b-8df6-488a-8212-b3d959fa6261 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:57.040779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfad2b-8df6-488a-8212-b3d959fa6261 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:57.047627 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac3c8b-1a45-44a2-913e-e1be41fc7ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:57.047627 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac3c8b-1a45-44a2-913e-e1be41fc7ce4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:57.047627 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac3c8b-1a45-44a2-913e-e1be41fc7ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:57.047627 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac3c8b-1a45-44a2-913e-e1be41fc7ce4 admin admin] PATH_INFO: `/v3/projects/901f131566c041a4b50a99260b7e5283/users/673b2c89adaa40c3b09a6c3231887890/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:57.063529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ac3c8b-1a45-44a2-913e-e1be41fc7ce4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=901f131566c041a4b50a99260b7e5283, user_id=673b2c89adaa40c3b09a6c3231887890, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:57.066301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ac3c8b-1a45-44a2-913e-e1be41fc7ce4 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:57.184717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 980/1979] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:57 2022] PUT /identity/v3/projects/901f131566c041a4b50a99260b7e5283/users/673b2c89adaa40c3b09a6c3231887890/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:54:57.207868 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff60bd-1d08-4579-9bf5-f2640d70480e admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:57.207868 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff60bd-1d08-4579-9bf5-f2640d70480e admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:57.218926 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac3c8b-1a45-44a2-913e-e1be41fc7ce4 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:57.267654 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71447ae1-ddb4-4c36-bf05-d547802ea14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:57.268425 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71447ae1-ddb4-4c36-bf05-d547802ea14c admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:57.268743 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71447ae1-ddb4-4c36-bf05-d547802ea14c admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:57.269072 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71447ae1-ddb4-4c36-bf05-d547802ea14c admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:57.269735 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71447ae1-ddb4-4c36-bf05-d547802ea14c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:57.271386 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71447ae1-ddb4-4c36-bf05-d547802ea14c admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:57.282904 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1000/1980] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:54:57.287515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71447ae1-ddb4-4c36-bf05-d547802ea14c admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:57.298215 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71447ae1-ddb4-4c36-bf05-d547802ea14c admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:57.305286 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291bf0c-ddb9-4259-9057-717ad7349f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:57.305875 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291bf0c-ddb9-4259-9057-717ad7349f30 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:57.306347 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac3c8b-1a45-44a2-913e-e1be41fc7ce4 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:57.306811 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291bf0c-ddb9-4259-9057-717ad7349f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:57.307102 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291bf0c-ddb9-4259-9057-717ad7349f30 admin admin] PATH_INFO: `/v3/projects/901f131566c041a4b50a99260b7e5283/users/673b2c89adaa40c3b09a6c3231887890/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:57.309196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e291bf0c-ddb9-4259-9057-717ad7349f30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=901f131566c041a4b50a99260b7e5283, user_id=673b2c89adaa40c3b09a6c3231887890, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:57.310355 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d187931-e590-4281-84d7-20104997585c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:57.310970 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d187931-e590-4281-84d7-20104997585c service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:57.311270 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d187931-e590-4281-84d7-20104997585c service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:57.311556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d187931-e590-4281-84d7-20104997585c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:57.312072 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e291bf0c-ddb9-4259-9057-717ad7349f30 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:57.313832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-3d187931-e590-4281-84d7-20104997585c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:57.327439 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1001/1981] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:54:57 2022] PUT /identity/v3/projects/901f131566c041a4b50a99260b7e5283/users/673b2c89adaa40c3b09a6c3231887890/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:54:57.337416 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5a417-8bc3-4d5f-9af9-3e26f41b77f6 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:57.337671 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5a417-8bc3-4d5f-9af9-3e26f41b77f6 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:57.337950 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5a417-8bc3-4d5f-9af9-3e26f41b77f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:57.375895 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-90d5a417-8bc3-4d5f-9af9-3e26f41b77f6 None None] MFA Rules not processed for user `673b2c89adaa40c3b09a6c3231887890`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:54:57.377195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-90d5a417-8bc3-4d5f-9af9-3e26f41b77f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:57.409518 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d187931-e590-4281-84d7-20104997585c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:57.411844 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d187931-e590-4281-84d7-20104997585c service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:57.421078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 981/1982] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:57 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:54:57.443469 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:57.455265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1002/1983] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:54:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:54:57.467558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d187931-e590-4281-84d7-20104997585c service cinder] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:54:57.469048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-3d187931-e590-4281-84d7-20104997585c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:54:57.480418 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d187931-e590-4281-84d7-20104997585c service cinder] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:54:57.528862 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f36327-b712-41ba-a2cc-ef94fb8909aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:54:57.529467 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f36327-b712-41ba-a2cc-ef94fb8909aa service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:54:57.529750 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f36327-b712-41ba-a2cc-ef94fb8909aa service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:54:57.530022 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f36327-b712-41ba-a2cc-ef94fb8909aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:54:57.561175 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:54:57.596046 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f36327-b712-41ba-a2cc-ef94fb8909aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:54:57.597999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f36327-b712-41ba-a2cc-ef94fb8909aa service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:54:57.606775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 982/1984] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:54:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:55:03.082151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d5a417-8bc3-4d5f-9af9-3e26f41b77f6 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:03.091757 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d5a417-8bc3-4d5f-9af9-3e26f41b77f6 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:03.095549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc60694-76fb-45e8-803d-2953d77d1a23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:03.096166 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc60694-76fb-45e8-803d-2953d77d1a23 service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:03.096445 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc60694-76fb-45e8-803d-2953d77d1a23 service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:03.096832 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc60694-76fb-45e8-803d-2953d77d1a23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:03.106013 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc60694-76fb-45e8-803d-2953d77d1a23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:03.108654 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc60694-76fb-45e8-803d-2953d77d1a23 service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:03.117599 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1003/1985] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:55:03 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:55:04.825443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f36327-b712-41ba-a2cc-ef94fb8909aa service cinder] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:04.826930 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-68f36327-b712-41ba-a2cc-ef94fb8909aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:04.899308 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f36327-b712-41ba-a2cc-ef94fb8909aa service cinder] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:04.900300 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77039d-316a-47c7-8a0d-f33395eebbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:04.901066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77039d-316a-47c7-8a0d-f33395eebbfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:04.901350 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77039d-316a-47c7-8a0d-f33395eebbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:04.901626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77039d-316a-47c7-8a0d-f33395eebbfa admin admin] PATH_INFO: `/v3/users/769d0c4198dd43b3861649b72a9a86b1` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:04.902972 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b77039d-316a-47c7-8a0d-f33395eebbfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=769d0c4198dd43b3861649b72a9a86b1)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:04.904950 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b77039d-316a-47c7-8a0d-f33395eebbfa admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:05.119631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-7b77039d-316a-47c7-8a0d-f33395eebbfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '769d0c4198dd43b3861649b72a9a86b1'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:05.132052 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-7b77039d-316a-47c7-8a0d-f33395eebbfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '769d0c4198dd43b3861649b72a9a86b1'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:05.218603 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-7b77039d-316a-47c7-8a0d-f33395eebbfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '769d0c4198dd43b3861649b72a9a86b1'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:05.219382 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-7b77039d-316a-47c7-8a0d-f33395eebbfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '769d0c4198dd43b3861649b72a9a86b1'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:05.220047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-7b77039d-316a-47c7-8a0d-f33395eebbfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '769d0c4198dd43b3861649b72a9a86b1'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:05.233338 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 983/1986] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:55:04 2022] DELETE /identity/v3/users/769d0c4198dd43b3861649b72a9a86b1 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:05.238316 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57f429-ad12-4c85-93d0-c04e7257e370 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:05.238880 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57f429-ad12-4c85-93d0-c04e7257e370 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:05.238880 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57f429-ad12-4c85-93d0-c04e7257e370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:05.258206 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-3f57f429-ad12-4c85-93d0-c04e7257e370 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:05.259407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-3f57f429-ad12-4c85-93d0-c04e7257e370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:05.319509 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1004/1987] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:05.329625 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77039d-316a-47c7-8a0d-f33395eebbfa admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:05.331071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-7b77039d-316a-47c7-8a0d-f33395eebbfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:05.396425 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:05.400671 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77039d-316a-47c7-8a0d-f33395eebbfa admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:05.404639 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f97598-a5fd-4c8d-bbe4-a217895e51c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:05.405196 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f97598-a5fd-4c8d-bbe4-a217895e51c6 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:05.405507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f97598-a5fd-4c8d-bbe4-a217895e51c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:05.405800 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f97598-a5fd-4c8d-bbe4-a217895e51c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:05.420124 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f97598-a5fd-4c8d-bbe4-a217895e51c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:05.421858 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f97598-a5fd-4c8d-bbe4-a217895e51c6 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:05.432408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 984/1988] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:55:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:55:05.916880 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57f429-ad12-4c85-93d0-c04e7257e370 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:05.918316 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-3f57f429-ad12-4c85-93d0-c04e7257e370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:05.932117 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57f429-ad12-4c85-93d0-c04e7257e370 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:05.938103 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bfe551-8f99-4149-ab80-f81c20e6b6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:05.938664 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfe551-8f99-4149-ab80-f81c20e6b6b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:05.939008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfe551-8f99-4149-ab80-f81c20e6b6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:05.939284 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfe551-8f99-4149-ab80-f81c20e6b6b4 admin admin] PATH_INFO: `/v3/projects/6e673709310b4a67b96a10ce79d83bf9` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:05.940644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bfe551-8f99-4149-ab80-f81c20e6b6b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e673709310b4a67b96a10ce79d83bf9)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:05.943287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bfe551-8f99-4149-ab80-f81c20e6b6b4 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:06.101029 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f0bfe551-8f99-4149-ab80-f81c20e6b6b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e673709310b4a67b96a10ce79d83bf9', 'request_id': 'req-f0bfe551-8f99-4149-ab80-f81c20e6b6b4'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:06.173892 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f0bfe551-8f99-4149-ab80-f81c20e6b6b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e673709310b4a67b96a10ce79d83bf9', 'request_id': 'req-f0bfe551-8f99-4149-ab80-f81c20e6b6b4'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:06.181090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f0bfe551-8f99-4149-ab80-f81c20e6b6b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e673709310b4a67b96a10ce79d83bf9', 'request_id': 'req-f0bfe551-8f99-4149-ab80-f81c20e6b6b4'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:06.195598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f0bfe551-8f99-4149-ab80-f81c20e6b6b4 admin admin] The token cache is being invalidate because project 6e673709310b4a67b96a10ce79d83bf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:55:06.195814 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f0bfe551-8f99-4149-ab80-f81c20e6b6b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:06.196466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f0bfe551-8f99-4149-ab80-f81c20e6b6b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:06.198254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1005/1989] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:55:05 2022] DELETE /identity/v3/projects/6e673709310b4a67b96a10ce79d83bf9 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:06.203687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f332d-4c21-4ae6-b62a-6d7337737d02 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:06.204214 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f332d-4c21-4ae6-b62a-6d7337737d02 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:06.204337 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f332d-4c21-4ae6-b62a-6d7337737d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:06.225459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-3b3f332d-4c21-4ae6-b62a-6d7337737d02 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:06.226677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-3b3f332d-4c21-4ae6-b62a-6d7337737d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:06.291752 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 985/1990] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:06.302266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582f116-08ce-4f55-8b34-fd403a90d134 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:06.304067 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582f116-08ce-4f55-8b34-fd403a90d134 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:06.304613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582f116-08ce-4f55-8b34-fd403a90d134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:06.328094 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-a582f116-08ce-4f55-8b34-fd403a90d134 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:06.329359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a582f116-08ce-4f55-8b34-fd403a90d134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:06.339488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1006/1991] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:06.343962 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f332d-4c21-4ae6-b62a-6d7337737d02 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:06.354187 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f332d-4c21-4ae6-b62a-6d7337737d02 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:06.358871 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3fc2e-cbdb-4d6b-b049-74e7fa82375c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:06.359495 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3fc2e-cbdb-4d6b-b049-74e7fa82375c admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:06.359783 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3fc2e-cbdb-4d6b-b049-74e7fa82375c admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:06.360061 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3fc2e-cbdb-4d6b-b049-74e7fa82375c admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:06.360755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3fc2e-cbdb-4d6b-b049-74e7fa82375c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:55:06.361063 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3fc2e-cbdb-4d6b-b049-74e7fa82375c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:06.363237 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3fc2e-cbdb-4d6b-b049-74e7fa82375c admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:06.375622 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 986/1992] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:55:06.380891 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582f116-08ce-4f55-8b34-fd403a90d134 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:06.390930 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582f116-08ce-4f55-8b34-fd403a90d134 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:06.395712 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef619ee-8e9d-482d-936a-2a466a4dd6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:06.396370 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef619ee-8e9d-482d-936a-2a466a4dd6d9 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:06.396777 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef619ee-8e9d-482d-936a-2a466a4dd6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:06.397053 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef619ee-8e9d-482d-936a-2a466a4dd6d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:06.397690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef619ee-8e9d-482d-936a-2a466a4dd6d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:06.399640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef619ee-8e9d-482d-936a-2a466a4dd6d9 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:06.410375 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:06.506815 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1007/1993] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:55:06 2022] POST /identity/v3/projects => generated 371 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:55:06.511335 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3fc2e-cbdb-4d6b-b049-74e7fa82375c admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:06.521439 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3fc2e-cbdb-4d6b-b049-74e7fa82375c admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:06.578674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb4a94-21fe-4420-b4b1-4b05ba8070a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:06.579282 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb4a94-21fe-4420-b4b1-4b05ba8070a2 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:06.579564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb4a94-21fe-4420-b4b1-4b05ba8070a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:06.579839 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb4a94-21fe-4420-b4b1-4b05ba8070a2 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:06.580455 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb4a94-21fe-4420-b4b1-4b05ba8070a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:06.582472 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb4a94-21fe-4420-b4b1-4b05ba8070a2 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:06.707518 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 987/1994] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:55:06 2022] POST /identity/v3/users => generated 286 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:55:06.712064 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef619ee-8e9d-482d-936a-2a466a4dd6d9 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:06.730590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef619ee-8e9d-482d-936a-2a466a4dd6d9 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:06.736641 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3236b-af36-409c-84fa-13bdbcdf72f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:06.738379 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3236b-af36-409c-84fa-13bdbcdf72f9 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:06.738796 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3236b-af36-409c-84fa-13bdbcdf72f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:06.739209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3236b-af36-409c-84fa-13bdbcdf72f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:06.740277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca3236b-af36-409c-84fa-13bdbcdf72f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:06.742192 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca3236b-af36-409c-84fa-13bdbcdf72f9 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:06.761060 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1008/1995] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 00:55:06.761626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb4a94-21fe-4420-b4b1-4b05ba8070a2 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:06.786410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb4a94-21fe-4420-b4b1-4b05ba8070a2 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:06.799903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05a607-4702-4716-828c-2b422e2d51fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:06.801321 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05a607-4702-4716-828c-2b422e2d51fa admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:06.801617 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05a607-4702-4716-828c-2b422e2d51fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:06.804123 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05a607-4702-4716-828c-2b422e2d51fa admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:06.804914 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05a607-4702-4716-828c-2b422e2d51fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:06.808179 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05a607-4702-4716-828c-2b422e2d51fa admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:06.836576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 988/1996] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:55:06.840986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3236b-af36-409c-84fa-13bdbcdf72f9 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:06.861069 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3236b-af36-409c-84fa-13bdbcdf72f9 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:06.862488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a72b87-f022-4e66-bb6f-d9f314b36306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:06.864215 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a72b87-f022-4e66-bb6f-d9f314b36306 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:06.864624 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a72b87-f022-4e66-bb6f-d9f314b36306 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:06.865005 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a72b87-f022-4e66-bb6f-d9f314b36306 admin admin] PATH_INFO: `/v3/projects/9b49ff217de14a2aae05d524128dec49/users/2f93774232c1479aa3e856bffb352140/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:06.885155 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a72b87-f022-4e66-bb6f-d9f314b36306 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b49ff217de14a2aae05d524128dec49, user_id=2f93774232c1479aa3e856bffb352140, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:06.887983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a72b87-f022-4e66-bb6f-d9f314b36306 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:07.006997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1009/1997] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:55:06 2022] PUT /identity/v3/projects/9b49ff217de14a2aae05d524128dec49/users/2f93774232c1479aa3e856bffb352140/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:07.009146 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed1fa9-b5a7-40e1-a675-e8ad15f99796 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:07.009446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed1fa9-b5a7-40e1-a675-e8ad15f99796 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:07.009722 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed1fa9-b5a7-40e1-a675-e8ad15f99796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:07.042038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-25ed1fa9-b5a7-40e1-a675-e8ad15f99796 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:07.043251 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-25ed1fa9-b5a7-40e1-a675-e8ad15f99796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:07.094967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:07.105226 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 989/1998] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:07.122197 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a72b87-f022-4e66-bb6f-d9f314b36306 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:07.124898 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-01a72b87-f022-4e66-bb6f-d9f314b36306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:07.164379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07678a-7c42-4483-86c9-1f1fe68c7f36 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:07.164702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07678a-7c42-4483-86c9-1f1fe68c7f36 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:07.164985 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07678a-7c42-4483-86c9-1f1fe68c7f36 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:07.166926 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 990/1999] 10.222.0.59 () {54 vars in 921 bytes} [Fri Aug 26 00:55:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:55:07.177244 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021c3ce-ef4a-45aa-b313-0227160bb9ff None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:07.178203 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021c3ce-ef4a-45aa-b313-0227160bb9ff None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:07.178684 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021c3ce-ef4a-45aa-b313-0227160bb9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:07.196376 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a72b87-f022-4e66-bb6f-d9f314b36306 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:07.199683 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-0021c3ce-ef4a-45aa-b313-0227160bb9ff None None] MFA Rules not processed for user `2876f4182f0949039b862f053533fe85`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:07.202182 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbaf666-8787-42e5-93ad-1af4f89f2945 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:07.202303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-0021c3ce-ef4a-45aa-b313-0227160bb9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:07.202405 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbaf666-8787-42e5-93ad-1af4f89f2945 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:07.202405 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbaf666-8787-42e5-93ad-1af4f89f2945 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:07.202405 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbaf666-8787-42e5-93ad-1af4f89f2945 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:07.213169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbaf666-8787-42e5-93ad-1af4f89f2945 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:07.214907 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbaf666-8787-42e5-93ad-1af4f89f2945 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:07.228569 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1010/2000] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:55:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:55:07.270543 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 991/2001] 10.222.0.59 () {58 vars in 1018 bytes} [Fri Aug 26 00:55:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:07.281823 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbaf666-8787-42e5-93ad-1af4f89f2945 service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:07.283206 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-adbaf666-8787-42e5-93ad-1af4f89f2945 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:07.301579 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbaf666-8787-42e5-93ad-1af4f89f2945 service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:07.307328 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb42b9c-af88-4db2-8dba-087f95a69a3a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2876f4182f0949039b862f053533fe85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:07.308008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb42b9c-af88-4db2-8dba-087f95a69a3a service placement] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:07.308255 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb42b9c-af88-4db2-8dba-087f95a69a3a service placement] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:07.308813 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb42b9c-af88-4db2-8dba-087f95a69a3a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:07.318277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb42b9c-af88-4db2-8dba-087f95a69a3a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:07.320027 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb42b9c-af88-4db2-8dba-087f95a69a3a service placement] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:07.332570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1011/2002] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:55:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:55:08.158208 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021c3ce-ef4a-45aa-b313-0227160bb9ff None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:08.159748 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-0021c3ce-ef4a-45aa-b313-0227160bb9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:08.173150 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021c3ce-ef4a-45aa-b313-0227160bb9ff None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:08.178292 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af94c06-a349-4479-9bce-da533479c13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:08.178963 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af94c06-a349-4479-9bce-da533479c13a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:08.183151 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af94c06-a349-4479-9bce-da533479c13a admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:08.183151 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af94c06-a349-4479-9bce-da533479c13a admin admin] PATH_INFO: `/v3/users/c535e265953c4604b536538705b8285d` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:08.183151 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af94c06-a349-4479-9bce-da533479c13a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c535e265953c4604b536538705b8285d)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:08.183638 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af94c06-a349-4479-9bce-da533479c13a admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:08.498171 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-4af94c06-a349-4479-9bce-da533479c13a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c535e265953c4604b536538705b8285d'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:08.510312 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-4af94c06-a349-4479-9bce-da533479c13a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c535e265953c4604b536538705b8285d'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:08.629959 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-4af94c06-a349-4479-9bce-da533479c13a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c535e265953c4604b536538705b8285d'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:08.638954 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-4af94c06-a349-4479-9bce-da533479c13a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c535e265953c4604b536538705b8285d'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:08.638954 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-4af94c06-a349-4479-9bce-da533479c13a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c535e265953c4604b536538705b8285d'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:08.651069 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 992/2003] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:55:08 2022] DELETE /identity/v3/users/c535e265953c4604b536538705b8285d => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:09.043409 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e5cdb-6daa-4fd4-aae4-6789a21a3e47 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:09.043928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e5cdb-6daa-4fd4-aae4-6789a21a3e47 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:09.044030 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e5cdb-6daa-4fd4-aae4-6789a21a3e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:09.067270 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af94c06-a349-4479-9bce-da533479c13a admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:09.068852 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-4af94c06-a349-4479-9bce-da533479c13a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:09.088240 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-917e5cdb-6daa-4fd4-aae4-6789a21a3e47 None None] MFA Rules not processed for user `2f93774232c1479aa3e856bffb352140`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:09.089779 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-917e5cdb-6daa-4fd4-aae4-6789a21a3e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:09.176850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1012/2004] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:09.191439 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e5cdb-6daa-4fd4-aae4-6789a21a3e47 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:09.193070 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-917e5cdb-6daa-4fd4-aae4-6789a21a3e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:09.203851 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af94c06-a349-4479-9bce-da533479c13a admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:09.215050 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ff0933-d717-4926-83d8-805908758fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:09.216654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ff0933-d717-4926-83d8-805908758fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:09.217248 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ff0933-d717-4926-83d8-805908758fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:09.217919 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ff0933-d717-4926-83d8-805908758fb6 admin admin] PATH_INFO: `/v3/projects/e9346a3c8fcd4b169a2a1f55d8cee2b0` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:09.219542 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ff0933-d717-4926-83d8-805908758fb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9346a3c8fcd4b169a2a1f55d8cee2b0)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:09.223265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ff0933-d717-4926-83d8-805908758fb6 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:09.234265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:09.272720 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:09.315574 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e5cdb-6daa-4fd4-aae4-6789a21a3e47 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:09.320300 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d97ade-9019-443a-ab2e-e47d332c7794 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:09.320992 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d97ade-9019-443a-ab2e-e47d332c7794 service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:09.321278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d97ade-9019-443a-ab2e-e47d332c7794 service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:09.321553 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d97ade-9019-443a-ab2e-e47d332c7794 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:09.332443 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d97ade-9019-443a-ab2e-e47d332c7794 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:09.334593 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d97ade-9019-443a-ab2e-e47d332c7794 service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:09.355098 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1013/2005] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:55:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:55:09.370777 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c4ff0933-d717-4926-83d8-805908758fb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9346a3c8fcd4b169a2a1f55d8cee2b0', 'request_id': 'req-c4ff0933-d717-4926-83d8-805908758fb6'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:09.450686 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c4ff0933-d717-4926-83d8-805908758fb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9346a3c8fcd4b169a2a1f55d8cee2b0', 'request_id': 'req-c4ff0933-d717-4926-83d8-805908758fb6'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:09.460794 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c4ff0933-d717-4926-83d8-805908758fb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9346a3c8fcd4b169a2a1f55d8cee2b0', 'request_id': 'req-c4ff0933-d717-4926-83d8-805908758fb6'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:09.481010 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c4ff0933-d717-4926-83d8-805908758fb6 admin admin] The token cache is being invalidate because project e9346a3c8fcd4b169a2a1f55d8cee2b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:55:09.482182 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c4ff0933-d717-4926-83d8-805908758fb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:09.484528 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c4ff0933-d717-4926-83d8-805908758fb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:09.489242 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 993/2006] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:55:09 2022] DELETE /identity/v3/projects/e9346a3c8fcd4b169a2a1f55d8cee2b0 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:55:09.493585 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d97ade-9019-443a-ab2e-e47d332c7794 service cinder] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:09.518524 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d97ade-9019-443a-ab2e-e47d332c7794 service cinder] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:09.545077 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ff0933-d717-4926-83d8-805908758fb6 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:09.546594 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-c4ff0933-d717-4926-83d8-805908758fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:09.625801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae041de9-a62b-4d6d-b628-7832197b5c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:09.627218 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae041de9-a62b-4d6d-b628-7832197b5c46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:09.627773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae041de9-a62b-4d6d-b628-7832197b5c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:09.628224 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae041de9-a62b-4d6d-b628-7832197b5c46 admin admin] PATH_INFO: `/v3/users/85778cd9234949fb8a8cecb9b01320a5` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:09.629694 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae041de9-a62b-4d6d-b628-7832197b5c46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85778cd9234949fb8a8cecb9b01320a5)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:09.640648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae041de9-a62b-4d6d-b628-7832197b5c46 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:09.674938 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ff0933-d717-4926-83d8-805908758fb6 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:09.679946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800d0d76-5663-4544-b0c7-966754886d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:09.680654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d0d76-5663-4544-b0c7-966754886d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:09.680943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d0d76-5663-4544-b0c7-966754886d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:09.681219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d0d76-5663-4544-b0c7-966754886d8e admin admin] PATH_INFO: `/v3/users/b7e80549a31b4a4c812f416aca6e2e74` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:09.682373 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800d0d76-5663-4544-b0c7-966754886d8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7e80549a31b4a4c812f416aca6e2e74)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:09.684298 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800d0d76-5663-4544-b0c7-966754886d8e admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:09.866724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ae041de9-a62b-4d6d-b628-7832197b5c46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85778cd9234949fb8a8cecb9b01320a5'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:09.867383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ae041de9-a62b-4d6d-b628-7832197b5c46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85778cd9234949fb8a8cecb9b01320a5'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:09.880219 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ae041de9-a62b-4d6d-b628-7832197b5c46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85778cd9234949fb8a8cecb9b01320a5'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:09.890501 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-800d0d76-5663-4544-b0c7-966754886d8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7e80549a31b4a4c812f416aca6e2e74'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:09.902990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-800d0d76-5663-4544-b0c7-966754886d8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7e80549a31b4a4c812f416aca6e2e74'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:09.955981 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ae041de9-a62b-4d6d-b628-7832197b5c46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85778cd9234949fb8a8cecb9b01320a5'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:09.957555 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ae041de9-a62b-4d6d-b628-7832197b5c46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85778cd9234949fb8a8cecb9b01320a5'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:09.976975 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1014/2007] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:55:09 2022] DELETE /identity/v3/users/85778cd9234949fb8a8cecb9b01320a5 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:09.995843 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-800d0d76-5663-4544-b0c7-966754886d8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7e80549a31b4a4c812f416aca6e2e74'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:09.996557 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-800d0d76-5663-4544-b0c7-966754886d8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7e80549a31b4a4c812f416aca6e2e74'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:09.997294 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-800d0d76-5663-4544-b0c7-966754886d8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7e80549a31b4a4c812f416aca6e2e74'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:10.013539 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 994/2008] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:55:09 2022] DELETE /identity/v3/users/b7e80549a31b4a4c812f416aca6e2e74 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:10.017953 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41cf06-4ee4-4b84-ad62-eeb41cf8753a None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:10.018242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41cf06-4ee4-4b84-ad62-eeb41cf8753a None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:10.018528 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41cf06-4ee4-4b84-ad62-eeb41cf8753a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:10.042680 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-9d41cf06-4ee4-4b84-ad62-eeb41cf8753a None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:10.044032 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-9d41cf06-4ee4-4b84-ad62-eeb41cf8753a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:10.099330 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:10.111778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1015/2009] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:10.122212 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800d0d76-5663-4544-b0c7-966754886d8e admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:10.124964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-800d0d76-5663-4544-b0c7-966754886d8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:10.146127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:10.186826 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800d0d76-5663-4544-b0c7-966754886d8e admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:10.190546 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a67d1b-d3ee-4689-8552-ec63561810ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:10.191104 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a67d1b-d3ee-4689-8552-ec63561810ad service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:10.191386 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a67d1b-d3ee-4689-8552-ec63561810ad service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:10.191666 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a67d1b-d3ee-4689-8552-ec63561810ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:10.200868 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a67d1b-d3ee-4689-8552-ec63561810ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:10.202377 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a67d1b-d3ee-4689-8552-ec63561810ad service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:10.213457 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 995/2010] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:55:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:55:10.254787 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41cf06-4ee4-4b84-ad62-eeb41cf8753a None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:10.256317 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-9d41cf06-4ee4-4b84-ad62-eeb41cf8753a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:10.270041 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41cf06-4ee4-4b84-ad62-eeb41cf8753a None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:10.285684 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b38e6f-b70d-4b90-9cae-b4b07d020e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:10.285684 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b38e6f-b70d-4b90-9cae-b4b07d020e20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:10.285684 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b38e6f-b70d-4b90-9cae-b4b07d020e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:10.285684 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b38e6f-b70d-4b90-9cae-b4b07d020e20 admin admin] PATH_INFO: `/v3/projects/4c5b35039a1846b8862f2f17449f18b9` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:10.286016 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b38e6f-b70d-4b90-9cae-b4b07d020e20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c5b35039a1846b8862f2f17449f18b9)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:10.286016 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b38e6f-b70d-4b90-9cae-b4b07d020e20 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:10.384420 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-10b38e6f-b70d-4b90-9cae-b4b07d020e20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c5b35039a1846b8862f2f17449f18b9', 'request_id': 'req-10b38e6f-b70d-4b90-9cae-b4b07d020e20'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:10.452663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-10b38e6f-b70d-4b90-9cae-b4b07d020e20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c5b35039a1846b8862f2f17449f18b9', 'request_id': 'req-10b38e6f-b70d-4b90-9cae-b4b07d020e20'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:10.460105 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-10b38e6f-b70d-4b90-9cae-b4b07d020e20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c5b35039a1846b8862f2f17449f18b9', 'request_id': 'req-10b38e6f-b70d-4b90-9cae-b4b07d020e20'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:10.472839 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-10b38e6f-b70d-4b90-9cae-b4b07d020e20 admin admin] The token cache is being invalidate because project 4c5b35039a1846b8862f2f17449f18b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:55:10.473273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-10b38e6f-b70d-4b90-9cae-b4b07d020e20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:10.474656 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-10b38e6f-b70d-4b90-9cae-b4b07d020e20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:10.476341 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1016/2011] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:55:10 2022] DELETE /identity/v3/projects/4c5b35039a1846b8862f2f17449f18b9 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:10.481161 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c294628-9e5f-43f0-a0a4-aa90f317ab63 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:10.490282 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c294628-9e5f-43f0-a0a4-aa90f317ab63 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:10.490282 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c294628-9e5f-43f0-a0a4-aa90f317ab63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:10.504074 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-0c294628-9e5f-43f0-a0a4-aa90f317ab63 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:10.505337 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-0c294628-9e5f-43f0-a0a4-aa90f317ab63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:10.568094 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 996/2012] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:55:10.575671 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f3a0f-d880-4a11-aa86-349dfd88b4cc None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:10.575992 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f3a0f-d880-4a11-aa86-349dfd88b4cc None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:10.576273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f3a0f-d880-4a11-aa86-349dfd88b4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:10.596536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-b76f3a0f-d880-4a11-aa86-349dfd88b4cc None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:10.597782 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-b76f3a0f-d880-4a11-aa86-349dfd88b4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:10.608848 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1017/2013] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:10.611995 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c294628-9e5f-43f0-a0a4-aa90f317ab63 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:10.622826 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c294628-9e5f-43f0-a0a4-aa90f317ab63 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:10.623636 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f3a0f-d880-4a11-aa86-349dfd88b4cc None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:10.625175 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-b76f3a0f-d880-4a11-aa86-349dfd88b4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:10.629193 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b461f-ed4f-47ac-8247-24fbb7861cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:10.629883 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b461f-ed4f-47ac-8247-24fbb7861cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:10.632570 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b461f-ed4f-47ac-8247-24fbb7861cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:10.633098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b461f-ed4f-47ac-8247-24fbb7861cd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:10.635264 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58b461f-ed4f-47ac-8247-24fbb7861cd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:55:10.635648 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58b461f-ed4f-47ac-8247-24fbb7861cd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:10.637717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58b461f-ed4f-47ac-8247-24fbb7861cd9 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:10.637909 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f3a0f-d880-4a11-aa86-349dfd88b4cc None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:10.642751 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bba01d-8d43-4341-bfd4-522ea86ff345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:10.643312 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bba01d-8d43-4341-bfd4-522ea86ff345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:10.643587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bba01d-8d43-4341-bfd4-522ea86ff345 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:10.643929 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bba01d-8d43-4341-bfd4-522ea86ff345 admin admin] PATH_INFO: `/v3/projects/a513e6b627b84b8a9e4fa81d1fd8fc4a` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:10.651584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bba01d-8d43-4341-bfd4-522ea86ff345 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a513e6b627b84b8a9e4fa81d1fd8fc4a)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:10.651584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bba01d-8d43-4341-bfd4-522ea86ff345 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:10.664531 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 997/2014] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:55:10.671558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b461f-ed4f-47ac-8247-24fbb7861cd9 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:10.690453 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b461f-ed4f-47ac-8247-24fbb7861cd9 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:10.696784 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3fc2c-9d1f-481b-afce-3db35ea7ab05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:10.698581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3fc2c-9d1f-481b-afce-3db35ea7ab05 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:10.698581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3fc2c-9d1f-481b-afce-3db35ea7ab05 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:10.698581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3fc2c-9d1f-481b-afce-3db35ea7ab05 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:10.703568 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a3fc2c-9d1f-481b-afce-3db35ea7ab05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:10.703568 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a3fc2c-9d1f-481b-afce-3db35ea7ab05 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:10.781517 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-07bba01d-8d43-4341-bfd4-522ea86ff345 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a513e6b627b84b8a9e4fa81d1fd8fc4a', 'request_id': 'req-07bba01d-8d43-4341-bfd4-522ea86ff345'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:10.797501 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 998/2015] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:55:10 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:55:10.801574 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3fc2c-9d1f-481b-afce-3db35ea7ab05 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:10.864601 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-07bba01d-8d43-4341-bfd4-522ea86ff345 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a513e6b627b84b8a9e4fa81d1fd8fc4a', 'request_id': 'req-07bba01d-8d43-4341-bfd4-522ea86ff345'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:10.869211 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3fc2c-9d1f-481b-afce-3db35ea7ab05 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:10.871190 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-07bba01d-8d43-4341-bfd4-522ea86ff345 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a513e6b627b84b8a9e4fa81d1fd8fc4a', 'request_id': 'req-07bba01d-8d43-4341-bfd4-522ea86ff345'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:10.876467 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd210b1a-5c1c-4329-9d4c-8d738602468f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:10.877106 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd210b1a-5c1c-4329-9d4c-8d738602468f admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:10.877396 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd210b1a-5c1c-4329-9d4c-8d738602468f admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:10.877671 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd210b1a-5c1c-4329-9d4c-8d738602468f admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:10.878938 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd210b1a-5c1c-4329-9d4c-8d738602468f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:10.881074 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd210b1a-5c1c-4329-9d4c-8d738602468f admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:10.883835 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-07bba01d-8d43-4341-bfd4-522ea86ff345 admin admin] The token cache is being invalidate because project a513e6b627b84b8a9e4fa81d1fd8fc4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:55:10.884127 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-07bba01d-8d43-4341-bfd4-522ea86ff345 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:10.884816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-07bba01d-8d43-4341-bfd4-522ea86ff345 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:10.886430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1018/2016] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:55:10 2022] DELETE /identity/v3/projects/a513e6b627b84b8a9e4fa81d1fd8fc4a => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:10.890767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bba01d-8d43-4341-bfd4-522ea86ff345 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:10.892084 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-07bba01d-8d43-4341-bfd4-522ea86ff345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:10.952965 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:10.958130 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bba01d-8d43-4341-bfd4-522ea86ff345 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:10.961227 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 999/2017] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:55:10 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:55:10.963284 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92235a8-4114-4625-b96b-f84b96778e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:10.964002 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92235a8-4114-4625-b96b-f84b96778e54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:10.964535 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92235a8-4114-4625-b96b-f84b96778e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:10.965217 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92235a8-4114-4625-b96b-f84b96778e54 admin admin] PATH_INFO: `/v3/users/673b2c89adaa40c3b09a6c3231887890` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:10.965899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd210b1a-5c1c-4329-9d4c-8d738602468f admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:10.966384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92235a8-4114-4625-b96b-f84b96778e54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=673b2c89adaa40c3b09a6c3231887890)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:10.967497 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-cd210b1a-5c1c-4329-9d4c-8d738602468f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:10.968418 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92235a8-4114-4625-b96b-f84b96778e54 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:10.980115 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd210b1a-5c1c-4329-9d4c-8d738602468f admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:10.987038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d2710-7a20-47e1-9b22-2aaf4c31ab70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:10.988311 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d2710-7a20-47e1-9b22-2aaf4c31ab70 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:10.988433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d2710-7a20-47e1-9b22-2aaf4c31ab70 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:10.989480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d2710-7a20-47e1-9b22-2aaf4c31ab70 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:10.989480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d2710-7a20-47e1-9b22-2aaf4c31ab70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:10.991005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d2710-7a20-47e1-9b22-2aaf4c31ab70 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:11.001223 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1000/2018] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:55:11.005721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d2710-7a20-47e1-9b22-2aaf4c31ab70 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:11.010722 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:11.016142 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d2710-7a20-47e1-9b22-2aaf4c31ab70 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:11.020889 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579f0e1-733d-49f6-b10e-89221a875661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:11.021477 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579f0e1-733d-49f6-b10e-89221a875661 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:11.021759 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579f0e1-733d-49f6-b10e-89221a875661 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:11.022034 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579f0e1-733d-49f6-b10e-89221a875661 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:11.022694 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b579f0e1-733d-49f6-b10e-89221a875661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:11.024256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b579f0e1-733d-49f6-b10e-89221a875661 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:11.034558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1001/2019] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 00:55:11.038997 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579f0e1-733d-49f6-b10e-89221a875661 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:11.048533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579f0e1-733d-49f6-b10e-89221a875661 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:11.054170 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f045b-4583-4215-ab69-8d7aa32f5190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:11.054841 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f045b-4583-4215-ab69-8d7aa32f5190 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:11.058411 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f045b-4583-4215-ab69-8d7aa32f5190 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:11.058411 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f045b-4583-4215-ab69-8d7aa32f5190 admin admin] PATH_INFO: `/v3/projects/1391afd5b05e43bea0d2c30e4bb11fa8/users/1f061f6ca0ee47d1b8b81653c792b48b/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:11.070468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f045b-4583-4215-ab69-8d7aa32f5190 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1391afd5b05e43bea0d2c30e4bb11fa8, user_id=1f061f6ca0ee47d1b8b81653c792b48b, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:11.073238 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f045b-4583-4215-ab69-8d7aa32f5190 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:11.150180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f92235a8-4114-4625-b96b-f84b96778e54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '673b2c89adaa40c3b09a6c3231887890'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:11.150857 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f92235a8-4114-4625-b96b-f84b96778e54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '673b2c89adaa40c3b09a6c3231887890'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:11.163859 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f92235a8-4114-4625-b96b-f84b96778e54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '673b2c89adaa40c3b09a6c3231887890'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:11.169194 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1002/2020] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:55:11 2022] PUT /identity/v3/projects/1391afd5b05e43bea0d2c30e4bb11fa8/users/1f061f6ca0ee47d1b8b81653c792b48b/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:55:11.176182 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa058e2-2c6d-46d2-88b0-8ae3ee238abc None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:11.176495 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa058e2-2c6d-46d2-88b0-8ae3ee238abc None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:11.176835 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa058e2-2c6d-46d2-88b0-8ae3ee238abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:11.198146 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-6fa058e2-2c6d-46d2-88b0-8ae3ee238abc None None] MFA Rules not processed for user `2f93774232c1479aa3e856bffb352140`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:11.199384 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-6fa058e2-2c6d-46d2-88b0-8ae3ee238abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:11.247276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f92235a8-4114-4625-b96b-f84b96778e54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '673b2c89adaa40c3b09a6c3231887890'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:11.248039 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f92235a8-4114-4625-b96b-f84b96778e54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '673b2c89adaa40c3b09a6c3231887890'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:11.254234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1003/2021] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:11.259470 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c8d87-9b75-46ef-ba3c-e7d74856961e None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:11.267953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c8d87-9b75-46ef-ba3c-e7d74856961e None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:11.267953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c8d87-9b75-46ef-ba3c-e7d74856961e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:11.268329 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1019/2022] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:55:10 2022] DELETE /identity/v3/users/673b2c89adaa40c3b09a6c3231887890 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:55:11.293952 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-292c8d87-9b75-46ef-ba3c-e7d74856961e None None] MFA Rules not processed for user `1f061f6ca0ee47d1b8b81653c792b48b`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:11.295155 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-292c8d87-9b75-46ef-ba3c-e7d74856961e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:11.363504 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1004/2023] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:11.367990 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92235a8-4114-4625-b96b-f84b96778e54 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:11.369447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f92235a8-4114-4625-b96b-f84b96778e54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:11.436070 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92235a8-4114-4625-b96b-f84b96778e54 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:11.443091 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1c9fb-fa7b-4c8d-bd76-73519319b1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:11.443574 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1c9fb-fa7b-4c8d-bd76-73519319b1b7 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:11.443819 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1c9fb-fa7b-4c8d-bd76-73519319b1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:11.444091 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1c9fb-fa7b-4c8d-bd76-73519319b1b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:11.445328 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1c9fb-fa7b-4c8d-bd76-73519319b1b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:11.446698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1c9fb-fa7b-4c8d-bd76-73519319b1b7 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:11.574719 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1020/2024] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:55:11 2022] POST /identity/v3/projects => generated 373 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:55:11.578446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c8d87-9b75-46ef-ba3c-e7d74856961e None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:11.591582 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c8d87-9b75-46ef-ba3c-e7d74856961e None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:11.663843 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38b96a-76a9-405b-8780-599c72a9a551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:11.664522 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38b96a-76a9-405b-8780-599c72a9a551 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:11.665569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38b96a-76a9-405b-8780-599c72a9a551 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:11.665569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38b96a-76a9-405b-8780-599c72a9a551 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:11.666116 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd38b96a-76a9-405b-8780-599c72a9a551 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:11.667607 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd38b96a-76a9-405b-8780-599c72a9a551 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:11.677993 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:11.762592 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1005/2025] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:55:11 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:55:11.766721 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1c9fb-fa7b-4c8d-bd76-73519319b1b7 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:11.778291 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1c9fb-fa7b-4c8d-bd76-73519319b1b7 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:11.784001 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce82c2d5-10c2-4ac1-9a74-84eb74edcf4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:11.784688 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82c2d5-10c2-4ac1-9a74-84eb74edcf4d admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:11.784971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82c2d5-10c2-4ac1-9a74-84eb74edcf4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:11.785240 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82c2d5-10c2-4ac1-9a74-84eb74edcf4d admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:11.789476 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce82c2d5-10c2-4ac1-9a74-84eb74edcf4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:11.789476 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce82c2d5-10c2-4ac1-9a74-84eb74edcf4d admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:11.799585 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1021/2026] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:55:11.804994 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38b96a-76a9-405b-8780-599c72a9a551 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:11.815254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38b96a-76a9-405b-8780-599c72a9a551 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:11.820640 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282d3a8-0e7b-4480-944f-07dd51e42624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:11.821308 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282d3a8-0e7b-4480-944f-07dd51e42624 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:11.821590 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282d3a8-0e7b-4480-944f-07dd51e42624 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:11.821862 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282d3a8-0e7b-4480-944f-07dd51e42624 admin admin] PATH_INFO: `/v3/projects/b2b7662695434c13ba1ba0b820f8f160/users/dc110963699e42079849e112e24fb074/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:11.835310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce82c2d5-10c2-4ac1-9a74-84eb74edcf4d admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:11.837231 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-ce82c2d5-10c2-4ac1-9a74-84eb74edcf4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:11.856274 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5282d3a8-0e7b-4480-944f-07dd51e42624 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2b7662695434c13ba1ba0b820f8f160, user_id=dc110963699e42079849e112e24fb074, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:11.858345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce82c2d5-10c2-4ac1-9a74-84eb74edcf4d admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:11.860576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5282d3a8-0e7b-4480-944f-07dd51e42624 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:11.873132 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15bbc6-d299-4560-af45-127da7446c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:11.873881 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15bbc6-d299-4560-af45-127da7446c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:11.874165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15bbc6-d299-4560-af45-127da7446c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:11.874436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15bbc6-d299-4560-af45-127da7446c4b admin admin] PATH_INFO: `/v3/projects/901f131566c041a4b50a99260b7e5283` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:11.875589 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15bbc6-d299-4560-af45-127da7446c4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=901f131566c041a4b50a99260b7e5283)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:11.877872 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15bbc6-d299-4560-af45-127da7446c4b admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:11.945821 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1006/2027] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:55:11 2022] PUT /identity/v3/projects/b2b7662695434c13ba1ba0b820f8f160/users/dc110963699e42079849e112e24fb074/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:11.950755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282d3a8-0e7b-4480-944f-07dd51e42624 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:12.007063 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:12.011012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-af15bbc6-d299-4560-af45-127da7446c4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '901f131566c041a4b50a99260b7e5283', 'request_id': 'req-af15bbc6-d299-4560-af45-127da7446c4b'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:12.054369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282d3a8-0e7b-4480-944f-07dd51e42624 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:12.104255 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-af15bbc6-d299-4560-af45-127da7446c4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '901f131566c041a4b50a99260b7e5283', 'request_id': 'req-af15bbc6-d299-4560-af45-127da7446c4b'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:12.110529 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-af15bbc6-d299-4560-af45-127da7446c4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '901f131566c041a4b50a99260b7e5283', 'request_id': 'req-af15bbc6-d299-4560-af45-127da7446c4b'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:12.123052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-af15bbc6-d299-4560-af45-127da7446c4b admin admin] The token cache is being invalidate because project 901f131566c041a4b50a99260b7e5283 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:55:12.123349 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-af15bbc6-d299-4560-af45-127da7446c4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:12.124000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-af15bbc6-d299-4560-af45-127da7446c4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:12.125384 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f437f1-84bb-4651-aabd-18e98024e708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:12.125961 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1022/2028] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:55:11 2022] DELETE /identity/v3/projects/901f131566c041a4b50a99260b7e5283 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:55:12.129421 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f437f1-84bb-4651-aabd-18e98024e708 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:12.129885 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f437f1-84bb-4651-aabd-18e98024e708 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:12.130236 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f437f1-84bb-4651-aabd-18e98024e708 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:12.131022 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f437f1-84bb-4651-aabd-18e98024e708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:12.132745 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f437f1-84bb-4651-aabd-18e98024e708 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:12.134316 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35f584-3446-4a61-99d2-5b9b37a30759 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:12.137120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35f584-3446-4a61-99d2-5b9b37a30759 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:12.137120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35f584-3446-4a61-99d2-5b9b37a30759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:12.143370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1007/2029] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 00:55:12.149026 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f437f1-84bb-4651-aabd-18e98024e708 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:12.150836 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-70f437f1-84bb-4651-aabd-18e98024e708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:12.162407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-8c35f584-3446-4a61-99d2-5b9b37a30759 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:12.163757 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-8c35f584-3446-4a61-99d2-5b9b37a30759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:12.169783 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f437f1-84bb-4651-aabd-18e98024e708 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:12.173250 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1023/2030] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:55:12.175462 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a66925-f936-41ce-adde-f00ec834842f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:12.176134 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a66925-f936-41ce-adde-f00ec834842f admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:12.176410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a66925-f936-41ce-adde-f00ec834842f admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:12.176719 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a66925-f936-41ce-adde-f00ec834842f admin admin] PATH_INFO: `/v3/projects/b2b7662695434c13ba1ba0b820f8f160/users/dc110963699e42079849e112e24fb074/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:12.178774 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a66925-f936-41ce-adde-f00ec834842f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2b7662695434c13ba1ba0b820f8f160, user_id=dc110963699e42079849e112e24fb074, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:12.181174 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f3c5b-204d-4337-badf-e457152662ed None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:12.181678 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f3c5b-204d-4337-badf-e457152662ed None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:12.181787 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a66925-f936-41ce-adde-f00ec834842f admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:12.182229 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f3c5b-204d-4337-badf-e457152662ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:12.198084 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1008/2031] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:55:12 2022] PUT /identity/v3/projects/b2b7662695434c13ba1ba0b820f8f160/users/dc110963699e42079849e112e24fb074/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:55:12.208063 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-fe1f3c5b-204d-4337-badf-e457152662ed None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:12.209268 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427bad7-0319-44b2-b304-035d31a5706f None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:12.209597 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-fe1f3c5b-204d-4337-badf-e457152662ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:12.209701 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427bad7-0319-44b2-b304-035d31a5706f None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:12.210841 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427bad7-0319-44b2-b304-035d31a5706f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:12.242125 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-d427bad7-0319-44b2-b304-035d31a5706f None None] MFA Rules not processed for user `dc110963699e42079849e112e24fb074`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:12.243366 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-d427bad7-0319-44b2-b304-035d31a5706f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:12.277457 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1024/2032] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:55:12.281850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f3c5b-204d-4337-badf-e457152662ed None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:12.291120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f3c5b-204d-4337-badf-e457152662ed None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:12.295857 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa8dc9-35a9-428e-95dc-3581c65923d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:12.296453 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa8dc9-35a9-428e-95dc-3581c65923d4 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:12.296763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa8dc9-35a9-428e-95dc-3581c65923d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:12.297036 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa8dc9-35a9-428e-95dc-3581c65923d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:12.297681 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aa8dc9-35a9-428e-95dc-3581c65923d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:55:12.297983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aa8dc9-35a9-428e-95dc-3581c65923d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:12.299212 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:12.299585 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aa8dc9-35a9-428e-95dc-3581c65923d4 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:12.310938 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1025/2033] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:55:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:55:12.316295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa8dc9-35a9-428e-95dc-3581c65923d4 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:12.322810 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1009/2034] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:55:12.326665 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa8dc9-35a9-428e-95dc-3581c65923d4 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:12.332923 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741f61f-e033-492e-94e7-a7cc477eca44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:12.333864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741f61f-e033-492e-94e7-a7cc477eca44 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:12.336490 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741f61f-e033-492e-94e7-a7cc477eca44 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:12.336490 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741f61f-e033-492e-94e7-a7cc477eca44 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:12.336653 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427bad7-0319-44b2-b304-035d31a5706f None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:12.336734 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741f61f-e033-492e-94e7-a7cc477eca44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:12.336839 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-d427bad7-0319-44b2-b304-035d31a5706f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:12.336928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741f61f-e033-492e-94e7-a7cc477eca44 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:12.392177 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427bad7-0319-44b2-b304-035d31a5706f None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:12.395997 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda34a5-c475-4637-8f44-cc555930f4fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:12.396650 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda34a5-c475-4637-8f44-cc555930f4fb service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:12.396931 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda34a5-c475-4637-8f44-cc555930f4fb service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:12.397206 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda34a5-c475-4637-8f44-cc555930f4fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:12.399078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-3bda34a5-c475-4637-8f44-cc555930f4fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:12.444777 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1026/2035] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:55:12 2022] POST /identity/v3/projects => generated 361 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:55:12.449464 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741f61f-e033-492e-94e7-a7cc477eca44 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:12.462779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda34a5-c475-4637-8f44-cc555930f4fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:12.464652 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda34a5-c475-4637-8f44-cc555930f4fb service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:12.478190 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1010/2036] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:55:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:55:12.526894 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741f61f-e033-492e-94e7-a7cc477eca44 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:12.534280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b3b81-63f2-4344-90fb-356f045c1282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:12.535403 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b3b81-63f2-4344-90fb-356f045c1282 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:12.535989 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b3b81-63f2-4344-90fb-356f045c1282 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:12.536600 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b3b81-63f2-4344-90fb-356f045c1282 admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:12.537437 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b3b81-63f2-4344-90fb-356f045c1282 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:12.541103 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b3b81-63f2-4344-90fb-356f045c1282 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:12.629889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1027/2037] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:55:12 2022] POST /identity/v3/users => generated 281 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:55:12.632466 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda34a5-c475-4637-8f44-cc555930f4fb service cinder] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:12.649768 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda34a5-c475-4637-8f44-cc555930f4fb service cinder] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:12.656851 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d7f79-df1a-4603-9373-1a704ca35f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:12.659612 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d7f79-df1a-4603-9373-1a704ca35f20 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:12.659994 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d7f79-df1a-4603-9373-1a704ca35f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:12.660434 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d7f79-df1a-4603-9373-1a704ca35f20 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:12.662505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86d7f79-df1a-4603-9373-1a704ca35f20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:12.664506 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86d7f79-df1a-4603-9373-1a704ca35f20 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:12.682078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1011/2038] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:55:12.682507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b3b81-63f2-4344-90fb-356f045c1282 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:12.697892 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b3b81-63f2-4344-90fb-356f045c1282 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:12.704114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c063e-c32e-4ccf-9453-812820c14347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:12.705344 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c063e-c32e-4ccf-9453-812820c14347 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:12.705631 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c063e-c32e-4ccf-9453-812820c14347 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:12.705906 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c063e-c32e-4ccf-9453-812820c14347 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:12.707035 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c063e-c32e-4ccf-9453-812820c14347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:12.708709 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c063e-c32e-4ccf-9453-812820c14347 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:12.719724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1028/2039] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:55:12.723962 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d7f79-df1a-4603-9373-1a704ca35f20 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:12.735045 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d7f79-df1a-4603-9373-1a704ca35f20 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:12.740347 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a4986-7490-4ecc-a585-944c9adbb0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:12.741019 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a4986-7490-4ecc-a585-944c9adbb0af admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:12.741304 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a4986-7490-4ecc-a585-944c9adbb0af admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:12.741578 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a4986-7490-4ecc-a585-944c9adbb0af admin admin] PATH_INFO: `/v3/projects/9f6bce3c334a4ad9a7ee23dc4b4916e1/users/9b09ce2cae7f41f68393ed5b2a7e3a9e/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:12.763033 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a4986-7490-4ecc-a585-944c9adbb0af admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f6bce3c334a4ad9a7ee23dc4b4916e1, user_id=9b09ce2cae7f41f68393ed5b2a7e3a9e, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:12.765775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a4986-7490-4ecc-a585-944c9adbb0af admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:12.853580 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1012/2040] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:55:12 2022] PUT /identity/v3/projects/9f6bce3c334a4ad9a7ee23dc4b4916e1/users/9b09ce2cae7f41f68393ed5b2a7e3a9e/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:12.866897 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530c952-dc25-4bcb-bc2d-acbed9448145 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:12.867185 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530c952-dc25-4bcb-bc2d-acbed9448145 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:12.867464 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530c952-dc25-4bcb-bc2d-acbed9448145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:12.899807 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:12.909257 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-4530c952-dc25-4bcb-bc2d-acbed9448145 None None] MFA Rules not processed for user `9b09ce2cae7f41f68393ed5b2a7e3a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:12.911744 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-4530c952-dc25-4bcb-bc2d-acbed9448145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:12.999259 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1029/2041] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:13.075190 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a4986-7490-4ecc-a585-944c9adbb0af admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:13.076897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-808a4986-7490-4ecc-a585-944c9adbb0af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:13.143456 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a4986-7490-4ecc-a585-944c9adbb0af admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:13.153169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d54b1-5853-450a-b140-3f2a9853f6fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:13.153889 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d54b1-5853-450a-b140-3f2a9853f6fd service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:13.154241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d54b1-5853-450a-b140-3f2a9853f6fd service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:13.154598 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d54b1-5853-450a-b140-3f2a9853f6fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:13.166747 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d54b1-5853-450a-b140-3f2a9853f6fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:13.168562 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d54b1-5853-450a-b140-3f2a9853f6fd service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:13.185596 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1013/2042] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:55:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:55:13.202474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530c952-dc25-4bcb-bc2d-acbed9448145 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:13.293534 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530c952-dc25-4bcb-bc2d-acbed9448145 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:13.307103 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753aa707-18bb-496e-818c-b8bfccc46f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:13.307103 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753aa707-18bb-496e-818c-b8bfccc46f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:13.307103 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753aa707-18bb-496e-818c-b8bfccc46f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:13.307103 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753aa707-18bb-496e-818c-b8bfccc46f62 admin admin] PATH_INFO: `/v3/users/9b09ce2cae7f41f68393ed5b2a7e3a9e` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:13.307405 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753aa707-18bb-496e-818c-b8bfccc46f62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b09ce2cae7f41f68393ed5b2a7e3a9e)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:13.307405 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753aa707-18bb-496e-818c-b8bfccc46f62 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:13.513018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-753aa707-18bb-496e-818c-b8bfccc46f62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b09ce2cae7f41f68393ed5b2a7e3a9e'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:13.513712 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-753aa707-18bb-496e-818c-b8bfccc46f62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b09ce2cae7f41f68393ed5b2a7e3a9e'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:13.526112 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-753aa707-18bb-496e-818c-b8bfccc46f62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b09ce2cae7f41f68393ed5b2a7e3a9e'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:13.618598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-753aa707-18bb-496e-818c-b8bfccc46f62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b09ce2cae7f41f68393ed5b2a7e3a9e'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:13.619417 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-753aa707-18bb-496e-818c-b8bfccc46f62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b09ce2cae7f41f68393ed5b2a7e3a9e'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:13.633620 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1030/2043] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:55:13 2022] DELETE /identity/v3/users/9b09ce2cae7f41f68393ed5b2a7e3a9e => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:13.638147 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac8c7a-0f0f-47f2-80aa-c56ed3f04aff None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:13.638637 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac8c7a-0f0f-47f2-80aa-c56ed3f04aff None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:13.639005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac8c7a-0f0f-47f2-80aa-c56ed3f04aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:13.647413 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:13.659134 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-afac8c7a-0f0f-47f2-80aa-c56ed3f04aff None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:13.661093 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-afac8c7a-0f0f-47f2-80aa-c56ed3f04aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:13.726022 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1014/2044] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:13.735944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753aa707-18bb-496e-818c-b8bfccc46f62 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:13.737492 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-753aa707-18bb-496e-818c-b8bfccc46f62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:13.758713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:13.804292 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753aa707-18bb-496e-818c-b8bfccc46f62 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:13.808453 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93ec26-bd85-45bd-99e5-b98a86082649 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:13.809165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93ec26-bd85-45bd-99e5-b98a86082649 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:13.809447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93ec26-bd85-45bd-99e5-b98a86082649 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:13.809720 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93ec26-bd85-45bd-99e5-b98a86082649 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:13.819486 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e93ec26-bd85-45bd-99e5-b98a86082649 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:13.821006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e93ec26-bd85-45bd-99e5-b98a86082649 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:13.830400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1031/2045] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:55:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:55:14.268513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac8c7a-0f0f-47f2-80aa-c56ed3f04aff None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:14.270322 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-afac8c7a-0f0f-47f2-80aa-c56ed3f04aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:14.288215 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac8c7a-0f0f-47f2-80aa-c56ed3f04aff None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:14.296415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec66c2-80b1-47a9-b71f-9ee91d2a6b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:14.300795 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec66c2-80b1-47a9-b71f-9ee91d2a6b55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:14.300795 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec66c2-80b1-47a9-b71f-9ee91d2a6b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:14.300795 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec66c2-80b1-47a9-b71f-9ee91d2a6b55 admin admin] PATH_INFO: `/v3/projects/9f6bce3c334a4ad9a7ee23dc4b4916e1` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:14.300795 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ec66c2-80b1-47a9-b71f-9ee91d2a6b55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f6bce3c334a4ad9a7ee23dc4b4916e1)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:14.302193 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ec66c2-80b1-47a9-b71f-9ee91d2a6b55 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:14.463331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c3ec66c2-80b1-47a9-b71f-9ee91d2a6b55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f6bce3c334a4ad9a7ee23dc4b4916e1', 'request_id': 'req-c3ec66c2-80b1-47a9-b71f-9ee91d2a6b55'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:14.538531 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c3ec66c2-80b1-47a9-b71f-9ee91d2a6b55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f6bce3c334a4ad9a7ee23dc4b4916e1', 'request_id': 'req-c3ec66c2-80b1-47a9-b71f-9ee91d2a6b55'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:14.545562 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c3ec66c2-80b1-47a9-b71f-9ee91d2a6b55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f6bce3c334a4ad9a7ee23dc4b4916e1', 'request_id': 'req-c3ec66c2-80b1-47a9-b71f-9ee91d2a6b55'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:14.557362 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c3ec66c2-80b1-47a9-b71f-9ee91d2a6b55 admin admin] The token cache is being invalidate because project 9f6bce3c334a4ad9a7ee23dc4b4916e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:55:14.557647 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c3ec66c2-80b1-47a9-b71f-9ee91d2a6b55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:14.558283 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c3ec66c2-80b1-47a9-b71f-9ee91d2a6b55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:14.559922 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1015/2046] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:55:14 2022] DELETE /identity/v3/projects/9f6bce3c334a4ad9a7ee23dc4b4916e1 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:15.821457 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93ec26-bd85-45bd-99e5-b98a86082649 service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:15.822982 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-4e93ec26-bd85-45bd-99e5-b98a86082649 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:15.836136 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93ec26-bd85-45bd-99e5-b98a86082649 service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:15.894808 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ad33e-fbe6-49d8-9d74-ffdb743b0c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:15.895369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ad33e-fbe6-49d8-9d74-ffdb743b0c43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:15.895675 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ad33e-fbe6-49d8-9d74-ffdb743b0c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:15.895932 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ad33e-fbe6-49d8-9d74-ffdb743b0c43 admin admin] PATH_INFO: `/v3/users/1f061f6ca0ee47d1b8b81653c792b48b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:15.897090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ad33e-fbe6-49d8-9d74-ffdb743b0c43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f061f6ca0ee47d1b8b81653c792b48b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:15.899032 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ad33e-fbe6-49d8-9d74-ffdb743b0c43 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:16.070959 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-8b5ad33e-fbe6-49d8-9d74-ffdb743b0c43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f061f6ca0ee47d1b8b81653c792b48b'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:16.071889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-8b5ad33e-fbe6-49d8-9d74-ffdb743b0c43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f061f6ca0ee47d1b8b81653c792b48b'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:16.090446 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-8b5ad33e-fbe6-49d8-9d74-ffdb743b0c43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f061f6ca0ee47d1b8b81653c792b48b'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:16.178900 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-8b5ad33e-fbe6-49d8-9d74-ffdb743b0c43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f061f6ca0ee47d1b8b81653c792b48b'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:16.179730 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-8b5ad33e-fbe6-49d8-9d74-ffdb743b0c43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f061f6ca0ee47d1b8b81653c792b48b'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:16.203177 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1032/2047] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:55:15 2022] DELETE /identity/v3/users/1f061f6ca0ee47d1b8b81653c792b48b => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:16.209301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c62b2-1b81-4afc-a8b5-8fbb2bb2c35d None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:16.210149 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c62b2-1b81-4afc-a8b5-8fbb2bb2c35d None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:16.210628 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c62b2-1b81-4afc-a8b5-8fbb2bb2c35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:16.233680 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-449c62b2-1b81-4afc-a8b5-8fbb2bb2c35d None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:16.235376 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-449c62b2-1b81-4afc-a8b5-8fbb2bb2c35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:16.308102 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1016/2048] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:16.328690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ad33e-fbe6-49d8-9d74-ffdb743b0c43 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:16.328690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-8b5ad33e-fbe6-49d8-9d74-ffdb743b0c43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:16.375533 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:16.394294 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ad33e-fbe6-49d8-9d74-ffdb743b0c43 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:16.398532 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a413f-1b9c-493c-a6ac-ae3d4f60256d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:16.399361 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a413f-1b9c-493c-a6ac-ae3d4f60256d service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:16.399915 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a413f-1b9c-493c-a6ac-ae3d4f60256d service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:16.400205 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a413f-1b9c-493c-a6ac-ae3d4f60256d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:16.427937 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11a413f-1b9c-493c-a6ac-ae3d4f60256d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:16.429626 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11a413f-1b9c-493c-a6ac-ae3d4f60256d service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:16.445891 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1033/2049] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:55:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:55:17.545724 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c62b2-1b81-4afc-a8b5-8fbb2bb2c35d None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:17.548392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-449c62b2-1b81-4afc-a8b5-8fbb2bb2c35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:17.562247 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c62b2-1b81-4afc-a8b5-8fbb2bb2c35d None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:17.568161 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d7df7-f3c3-43e2-8065-a209aeb19579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:17.569015 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d7df7-f3c3-43e2-8065-a209aeb19579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:17.569153 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d7df7-f3c3-43e2-8065-a209aeb19579 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:17.569425 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d7df7-f3c3-43e2-8065-a209aeb19579 admin admin] PATH_INFO: `/v3/projects/1391afd5b05e43bea0d2c30e4bb11fa8` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:17.570631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d7df7-f3c3-43e2-8065-a209aeb19579 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1391afd5b05e43bea0d2c30e4bb11fa8)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:17.573416 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d7df7-f3c3-43e2-8065-a209aeb19579 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:17.600682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:17.725218 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-bf6d7df7-f3c3-43e2-8065-a209aeb19579 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1391afd5b05e43bea0d2c30e4bb11fa8', 'request_id': 'req-bf6d7df7-f3c3-43e2-8065-a209aeb19579'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:17.806808 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-bf6d7df7-f3c3-43e2-8065-a209aeb19579 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1391afd5b05e43bea0d2c30e4bb11fa8', 'request_id': 'req-bf6d7df7-f3c3-43e2-8065-a209aeb19579'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:17.824088 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-bf6d7df7-f3c3-43e2-8065-a209aeb19579 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1391afd5b05e43bea0d2c30e4bb11fa8', 'request_id': 'req-bf6d7df7-f3c3-43e2-8065-a209aeb19579'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:17.856637 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-bf6d7df7-f3c3-43e2-8065-a209aeb19579 admin admin] The token cache is being invalidate because project 1391afd5b05e43bea0d2c30e4bb11fa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:55:17.856637 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-bf6d7df7-f3c3-43e2-8065-a209aeb19579 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:17.857606 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-bf6d7df7-f3c3-43e2-8065-a209aeb19579 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:17.860042 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1017/2050] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:55:17 2022] DELETE /identity/v3/projects/1391afd5b05e43bea0d2c30e4bb11fa8 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:17.865830 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a413f-1b9c-493c-a6ac-ae3d4f60256d service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:17.880798 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a413f-1b9c-493c-a6ac-ae3d4f60256d service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:17.994736 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187af6d9-4736-448d-95e9-d14a3c8d263a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:17.995548 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187af6d9-4736-448d-95e9-d14a3c8d263a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:17.995965 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187af6d9-4736-448d-95e9-d14a3c8d263a admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:17.996311 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187af6d9-4736-448d-95e9-d14a3c8d263a admin admin] PATH_INFO: `/v3/users/dc110963699e42079849e112e24fb074` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:17.998806 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187af6d9-4736-448d-95e9-d14a3c8d263a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc110963699e42079849e112e24fb074)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:18.005738 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187af6d9-4736-448d-95e9-d14a3c8d263a admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:18.236199 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-187af6d9-4736-448d-95e9-d14a3c8d263a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc110963699e42079849e112e24fb074'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:18.237783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-187af6d9-4736-448d-95e9-d14a3c8d263a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc110963699e42079849e112e24fb074'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:18.252326 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-187af6d9-4736-448d-95e9-d14a3c8d263a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc110963699e42079849e112e24fb074'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:18.350130 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-187af6d9-4736-448d-95e9-d14a3c8d263a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc110963699e42079849e112e24fb074'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:18.350807 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-187af6d9-4736-448d-95e9-d14a3c8d263a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc110963699e42079849e112e24fb074'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:18.365564 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1034/2051] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:55:17 2022] DELETE /identity/v3/users/dc110963699e42079849e112e24fb074 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:19.080677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d7df7-f3c3-43e2-8065-a209aeb19579 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:19.084228 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-bf6d7df7-f3c3-43e2-8065-a209aeb19579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:19.107799 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d7df7-f3c3-43e2-8065-a209aeb19579 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:19.264181 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f4a25-1420-432e-b542-ddd3373f3b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:19.268677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f4a25-1420-432e-b542-ddd3373f3b1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:19.269346 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f4a25-1420-432e-b542-ddd3373f3b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:19.269863 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f4a25-1420-432e-b542-ddd3373f3b1b admin admin] PATH_INFO: `/v3/projects/b2b7662695434c13ba1ba0b820f8f160` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:19.272208 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f4a25-1420-432e-b542-ddd3373f3b1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2b7662695434c13ba1ba0b820f8f160)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:19.276539 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f4a25-1420-432e-b542-ddd3373f3b1b admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:19.450705 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1e9f4a25-1420-432e-b542-ddd3373f3b1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2b7662695434c13ba1ba0b820f8f160', 'request_id': 'req-1e9f4a25-1420-432e-b542-ddd3373f3b1b'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:19.590024 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1e9f4a25-1420-432e-b542-ddd3373f3b1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2b7662695434c13ba1ba0b820f8f160', 'request_id': 'req-1e9f4a25-1420-432e-b542-ddd3373f3b1b'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:19.599499 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1e9f4a25-1420-432e-b542-ddd3373f3b1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2b7662695434c13ba1ba0b820f8f160', 'request_id': 'req-1e9f4a25-1420-432e-b542-ddd3373f3b1b'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:19.610570 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1e9f4a25-1420-432e-b542-ddd3373f3b1b admin admin] The token cache is being invalidate because project b2b7662695434c13ba1ba0b820f8f160 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:55:19.610857 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1e9f4a25-1420-432e-b542-ddd3373f3b1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:19.612193 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1e9f4a25-1420-432e-b542-ddd3373f3b1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:19.614589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1018/2052] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:55:19 2022] DELETE /identity/v3/projects/b2b7662695434c13ba1ba0b820f8f160 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:19.622189 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794cd9c7-77c1-4746-939b-7daebd9e60c9 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:19.622968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794cd9c7-77c1-4746-939b-7daebd9e60c9 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:19.623433 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794cd9c7-77c1-4746-939b-7daebd9e60c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:19.651520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-794cd9c7-77c1-4746-939b-7daebd9e60c9 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:19.651520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-794cd9c7-77c1-4746-939b-7daebd9e60c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:19.742812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1035/2053] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:19.750866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d36842-ce40-4705-bb66-17b5f6ef6bbd None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:19.751363 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d36842-ce40-4705-bb66-17b5f6ef6bbd None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:19.751788 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d36842-ce40-4705-bb66-17b5f6ef6bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:19.773881 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-06d36842-ce40-4705-bb66-17b5f6ef6bbd None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:19.775227 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-06d36842-ce40-4705-bb66-17b5f6ef6bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:19.786137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1019/2054] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:19.791044 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794cd9c7-77c1-4746-939b-7daebd9e60c9 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:19.796342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:19.803167 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794cd9c7-77c1-4746-939b-7daebd9e60c9 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:19.809035 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ad5ea-b307-4945-ae94-05a6a7c27418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:19.809768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ad5ea-b307-4945-ae94-05a6a7c27418 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:19.810146 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ad5ea-b307-4945-ae94-05a6a7c27418 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:19.810672 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ad5ea-b307-4945-ae94-05a6a7c27418 admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:19.811442 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ad5ea-b307-4945-ae94-05a6a7c27418 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:55:19.811893 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ad5ea-b307-4945-ae94-05a6a7c27418 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:19.813791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ad5ea-b307-4945-ae94-05a6a7c27418 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:19.828771 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1036/2055] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:55:19.833055 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d36842-ce40-4705-bb66-17b5f6ef6bbd None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:19.844862 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d36842-ce40-4705-bb66-17b5f6ef6bbd None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:19.858494 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ad5ea-b307-4945-ae94-05a6a7c27418 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:19.860955 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-3d4ad5ea-b307-4945-ae94-05a6a7c27418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:19.865924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8134c0b4-d6ce-40e8-a96f-3fc30fc1b0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:19.866811 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134c0b4-d6ce-40e8-a96f-3fc30fc1b0e0 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:19.867213 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134c0b4-d6ce-40e8-a96f-3fc30fc1b0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:19.868228 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134c0b4-d6ce-40e8-a96f-3fc30fc1b0e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:19.869139 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8134c0b4-d6ce-40e8-a96f-3fc30fc1b0e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:19.871161 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8134c0b4-d6ce-40e8-a96f-3fc30fc1b0e0 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:19.878717 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ad5ea-b307-4945-ae94-05a6a7c27418 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:19.886672 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7daec3-3796-4c63-b17c-7552f9836164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:19.886900 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7daec3-3796-4c63-b17c-7552f9836164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:19.887339 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7daec3-3796-4c63-b17c-7552f9836164 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:19.887625 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7daec3-3796-4c63-b17c-7552f9836164 admin admin] PATH_INFO: `/v3/users/e6867be02ee04d22819849a576c13fd5` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:19.888889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7daec3-3796-4c63-b17c-7552f9836164 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6867be02ee04d22819849a576c13fd5)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:19.896006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7daec3-3796-4c63-b17c-7552f9836164 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:19.968865 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1020/2056] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:55:19 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:55:19.974072 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8134c0b4-d6ce-40e8-a96f-3fc30fc1b0e0 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:19.987876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:20.047257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8134c0b4-d6ce-40e8-a96f-3fc30fc1b0e0 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:20.052082 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea72458-ff72-4566-b47b-e54b6ac2b9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:20.052727 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea72458-ff72-4566-b47b-e54b6ac2b9f0 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:20.053008 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea72458-ff72-4566-b47b-e54b6ac2b9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:20.053281 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea72458-ff72-4566-b47b-e54b6ac2b9f0 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:20.053926 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea72458-ff72-4566-b47b-e54b6ac2b9f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:20.055814 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea72458-ff72-4566-b47b-e54b6ac2b9f0 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:20.070095 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-0a7daec3-3796-4c63-b17c-7552f9836164 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6867be02ee04d22819849a576c13fd5'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:20.071119 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-0a7daec3-3796-4c63-b17c-7552f9836164 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6867be02ee04d22819849a576c13fd5'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:20.084629 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-0a7daec3-3796-4c63-b17c-7552f9836164 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6867be02ee04d22819849a576c13fd5'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:20.121881 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1021/2057] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:55:19 2022] POST /identity/v3/users => generated 292 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:55:20.123269 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea72458-ff72-4566-b47b-e54b6ac2b9f0 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:20.125007 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-5ea72458-ff72-4566-b47b-e54b6ac2b9f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:20.132725 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-0a7daec3-3796-4c63-b17c-7552f9836164 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6867be02ee04d22819849a576c13fd5'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:20.133415 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-0a7daec3-3796-4c63-b17c-7552f9836164 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6867be02ee04d22819849a576c13fd5'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:20.138084 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea72458-ff72-4566-b47b-e54b6ac2b9f0 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:20.143050 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb3aac-27a6-45d8-acb3-48600757e37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:20.144202 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb3aac-27a6-45d8-acb3-48600757e37b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:20.144509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb3aac-27a6-45d8-acb3-48600757e37b admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:20.144823 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb3aac-27a6-45d8-acb3-48600757e37b admin admin] PATH_INFO: `/v3/users/1a425027e59242bba6b05407d19cac82` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:20.146031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb3aac-27a6-45d8-acb3-48600757e37b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a425027e59242bba6b05407d19cac82)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:20.147096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1037/2058] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:55:19 2022] DELETE /identity/v3/users/e6867be02ee04d22819849a576c13fd5 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:20.150317 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7daec3-3796-4c63-b17c-7552f9836164 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:20.150431 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb3aac-27a6-45d8-acb3-48600757e37b admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:20.150523 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-0a7daec3-3796-4c63-b17c-7552f9836164 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:20.217192 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7daec3-3796-4c63-b17c-7552f9836164 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:20.222698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd361ce-7b8c-4fbe-bf75-e835e79c882a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:20.223303 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd361ce-7b8c-4fbe-bf75-e835e79c882a admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:20.223576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd361ce-7b8c-4fbe-bf75-e835e79c882a admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:20.223841 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd361ce-7b8c-4fbe-bf75-e835e79c882a admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:20.224496 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd361ce-7b8c-4fbe-bf75-e835e79c882a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:20.226147 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd361ce-7b8c-4fbe-bf75-e835e79c882a admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:20.236295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1038/2059] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:55:20.238130 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f54a5-9be1-41d3-a430-cdb0479154c3 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:20.239430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f54a5-9be1-41d3-a430-cdb0479154c3 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:20.239711 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f54a5-9be1-41d3-a430-cdb0479154c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:20.258090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-148f54a5-9be1-41d3-a430-cdb0479154c3 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:20.259487 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-148f54a5-9be1-41d3-a430-cdb0479154c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:20.267296 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1039/2060] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:20.268228 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f54a5-9be1-41d3-a430-cdb0479154c3 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:20.279591 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f54a5-9be1-41d3-a430-cdb0479154c3 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:20.280641 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f5eb3aac-27a6-45d8-acb3-48600757e37b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a425027e59242bba6b05407d19cac82'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:20.285015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928cf1b5-d6b9-4c91-ade5-29b57f9a2d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:20.285658 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928cf1b5-d6b9-4c91-ade5-29b57f9a2d63 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:20.285935 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928cf1b5-d6b9-4c91-ade5-29b57f9a2d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:20.286227 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928cf1b5-d6b9-4c91-ade5-29b57f9a2d63 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:20.287120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928cf1b5-d6b9-4c91-ade5-29b57f9a2d63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:20.288764 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928cf1b5-d6b9-4c91-ade5-29b57f9a2d63 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:20.293084 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f5eb3aac-27a6-45d8-acb3-48600757e37b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a425027e59242bba6b05407d19cac82'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:20.298673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1040/2061] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:55:20.305010 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928cf1b5-d6b9-4c91-ade5-29b57f9a2d63 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:20.305010 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-928cf1b5-d6b9-4c91-ade5-29b57f9a2d63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:20.343570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:20.355053 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f5eb3aac-27a6-45d8-acb3-48600757e37b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a425027e59242bba6b05407d19cac82'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:20.355734 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f5eb3aac-27a6-45d8-acb3-48600757e37b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a425027e59242bba6b05407d19cac82'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:20.356900 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f5eb3aac-27a6-45d8-acb3-48600757e37b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a425027e59242bba6b05407d19cac82'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:20.357323 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928cf1b5-d6b9-4c91-ade5-29b57f9a2d63 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:20.361756 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e687135-28db-4a86-af31-173158b034fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:20.363296 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e687135-28db-4a86-af31-173158b034fe service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:20.363560 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e687135-28db-4a86-af31-173158b034fe service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:20.363854 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e687135-28db-4a86-af31-173158b034fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:20.365823 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-8e687135-28db-4a86-af31-173158b034fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:20.370484 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1022/2062] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:55:20 2022] DELETE /identity/v3/users/1a425027e59242bba6b05407d19cac82 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:20.371338 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb3aac-27a6-45d8-acb3-48600757e37b admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:20.379900 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-f5eb3aac-27a6-45d8-acb3-48600757e37b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:20.381905 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e687135-28db-4a86-af31-173158b034fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:20.383353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e687135-28db-4a86-af31-173158b034fe service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:20.394171 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1041/2063] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:55:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:55:20.460750 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb3aac-27a6-45d8-acb3-48600757e37b admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:20.466000 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d405a4-2a3e-42c6-b0e2-5057f82005f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:20.466945 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d405a4-2a3e-42c6-b0e2-5057f82005f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:20.467541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d405a4-2a3e-42c6-b0e2-5057f82005f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:20.467649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d405a4-2a3e-42c6-b0e2-5057f82005f7 admin admin] PATH_INFO: `/v3/projects/31471c7bbea34dc0b250e444c70f9baa/users/cfd8cabf83b54550a8f816a108a56fa8/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:20.487983 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d405a4-2a3e-42c6-b0e2-5057f82005f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31471c7bbea34dc0b250e444c70f9baa, user_id=cfd8cabf83b54550a8f816a108a56fa8, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:20.490949 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d405a4-2a3e-42c6-b0e2-5057f82005f7 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:20.596348 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:20.602748 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1023/2064] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:55:20 2022] PUT /identity/v3/projects/31471c7bbea34dc0b250e444c70f9baa/users/cfd8cabf83b54550a8f816a108a56fa8/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:20.614316 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9adbbd-677b-4d98-a9e7-47135d784f69 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:20.614607 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9adbbd-677b-4d98-a9e7-47135d784f69 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:20.614882 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9adbbd-677b-4d98-a9e7-47135d784f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:20.642681 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-fb9adbbd-677b-4d98-a9e7-47135d784f69 None None] MFA Rules not processed for user `cfd8cabf83b54550a8f816a108a56fa8`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:20.644515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-fb9adbbd-677b-4d98-a9e7-47135d784f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:20.711287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1042/2065] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:20.715732 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d405a4-2a3e-42c6-b0e2-5057f82005f7 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:20.744557 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9adbbd-677b-4d98-a9e7-47135d784f69 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:20.745774 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-fb9adbbd-677b-4d98-a9e7-47135d784f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:20.812460 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d405a4-2a3e-42c6-b0e2-5057f82005f7 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:20.817966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547b80f-5562-4e94-a29d-70d6a516696c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:20.819720 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547b80f-5562-4e94-a29d-70d6a516696c admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:20.819720 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547b80f-5562-4e94-a29d-70d6a516696c admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:20.819720 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547b80f-5562-4e94-a29d-70d6a516696c admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:20.820065 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547b80f-5562-4e94-a29d-70d6a516696c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:20.822518 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547b80f-5562-4e94-a29d-70d6a516696c admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:20.827012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9adbbd-677b-4d98-a9e7-47135d784f69 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:20.832644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412f686-9a84-465b-bbce-8d48647a68d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:20.833660 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412f686-9a84-465b-bbce-8d48647a68d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:20.836905 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412f686-9a84-465b-bbce-8d48647a68d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:20.836905 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412f686-9a84-465b-bbce-8d48647a68d9 admin admin] PATH_INFO: `/v3/projects/032ed1b5aea842edb0bcae9d23810786` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:20.836905 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412f686-9a84-465b-bbce-8d48647a68d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=032ed1b5aea842edb0bcae9d23810786)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:20.839509 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412f686-9a84-465b-bbce-8d48647a68d9 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:20.894944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1024/2066] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:55:20 2022] POST /identity/v3/projects => generated 385 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:55:20.899058 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547b80f-5562-4e94-a29d-70d6a516696c admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:20.923006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-4412f686-9a84-465b-bbce-8d48647a68d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '032ed1b5aea842edb0bcae9d23810786', 'request_id': 'req-4412f686-9a84-465b-bbce-8d48647a68d9'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:20.968370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547b80f-5562-4e94-a29d-70d6a516696c admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:21.001127 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-4412f686-9a84-465b-bbce-8d48647a68d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '032ed1b5aea842edb0bcae9d23810786', 'request_id': 'req-4412f686-9a84-465b-bbce-8d48647a68d9'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:21.007891 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-4412f686-9a84-465b-bbce-8d48647a68d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '032ed1b5aea842edb0bcae9d23810786', 'request_id': 'req-4412f686-9a84-465b-bbce-8d48647a68d9'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:21.019428 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-4412f686-9a84-465b-bbce-8d48647a68d9 admin admin] The token cache is being invalidate because project 032ed1b5aea842edb0bcae9d23810786 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:55:21.019716 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-4412f686-9a84-465b-bbce-8d48647a68d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:21.020517 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-4412f686-9a84-465b-bbce-8d48647a68d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:21.022100 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1043/2067] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:55:20 2022] DELETE /identity/v3/projects/032ed1b5aea842edb0bcae9d23810786 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:55:21.023074 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412f686-9a84-465b-bbce-8d48647a68d9 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:21.024799 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-4412f686-9a84-465b-bbce-8d48647a68d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:21.031081 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:21.037983 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0dcd0d-b3a7-4bd6-9d12-528b0259ca85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:21.038608 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0dcd0d-b3a7-4bd6-9d12-528b0259ca85 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:21.038941 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0dcd0d-b3a7-4bd6-9d12-528b0259ca85 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:21.039219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0dcd0d-b3a7-4bd6-9d12-528b0259ca85 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:21.039853 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0dcd0d-b3a7-4bd6-9d12-528b0259ca85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:21.041983 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0dcd0d-b3a7-4bd6-9d12-528b0259ca85 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:21.090342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412f686-9a84-465b-bbce-8d48647a68d9 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:21.095555 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5702f2af-c364-48e9-ba54-cea807ecdef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:21.096206 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702f2af-c364-48e9-ba54-cea807ecdef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:21.096508 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702f2af-c364-48e9-ba54-cea807ecdef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:21.097347 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702f2af-c364-48e9-ba54-cea807ecdef3 admin admin] PATH_INFO: `/v3/projects/dce7a2fe13bf4130a256418df02eab16` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:21.098392 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5702f2af-c364-48e9-ba54-cea807ecdef3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dce7a2fe13bf4130a256418df02eab16)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:21.100287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5702f2af-c364-48e9-ba54-cea807ecdef3 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:21.116433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1025/2068] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:55:20 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:55:21.117888 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3abb1c-f544-431a-bde9-cc4a5ba6d7dc None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:21.118180 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3abb1c-f544-431a-bde9-cc4a5ba6d7dc None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:21.118452 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3abb1c-f544-431a-bde9-cc4a5ba6d7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:21.137780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-3d3abb1c-f544-431a-bde9-cc4a5ba6d7dc None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:21.138979 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-3d3abb1c-f544-431a-bde9-cc4a5ba6d7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:21.146932 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1026/2069] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:21.147783 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3abb1c-f544-431a-bde9-cc4a5ba6d7dc None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:21.149045 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-3d3abb1c-f544-431a-bde9-cc4a5ba6d7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:21.161204 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3abb1c-f544-431a-bde9-cc4a5ba6d7dc None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:21.166055 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f48c4-21b8-47db-86d4-ec9351e9f9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:21.166647 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f48c4-21b8-47db-86d4-ec9351e9f9bd admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:21.166960 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f48c4-21b8-47db-86d4-ec9351e9f9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:21.167200 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f48c4-21b8-47db-86d4-ec9351e9f9bd admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:21.167837 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85f48c4-21b8-47db-86d4-ec9351e9f9bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:21.169462 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85f48c4-21b8-47db-86d4-ec9351e9f9bd admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:21.186302 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1027/2070] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:55:21.186302 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45deac3e-7117-402f-b049-e15792f37e28 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:21.186302 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45deac3e-7117-402f-b049-e15792f37e28 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:21.186302 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45deac3e-7117-402f-b049-e15792f37e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:21.193078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:21.196436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-5702f2af-c364-48e9-ba54-cea807ecdef3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dce7a2fe13bf4130a256418df02eab16', 'request_id': 'req-5702f2af-c364-48e9-ba54-cea807ecdef3'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:21.209344 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-45deac3e-7117-402f-b049-e15792f37e28 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:21.210556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-45deac3e-7117-402f-b049-e15792f37e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:21.253515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-5702f2af-c364-48e9-ba54-cea807ecdef3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dce7a2fe13bf4130a256418df02eab16', 'request_id': 'req-5702f2af-c364-48e9-ba54-cea807ecdef3'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:21.261426 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-5702f2af-c364-48e9-ba54-cea807ecdef3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dce7a2fe13bf4130a256418df02eab16', 'request_id': 'req-5702f2af-c364-48e9-ba54-cea807ecdef3'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:21.274133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1028/2071] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:21.274976 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45deac3e-7117-402f-b049-e15792f37e28 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:21.276298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-5702f2af-c364-48e9-ba54-cea807ecdef3 admin admin] The token cache is being invalidate because project dce7a2fe13bf4130a256418df02eab16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:55:21.276713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-5702f2af-c364-48e9-ba54-cea807ecdef3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:21.277728 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-5702f2af-c364-48e9-ba54-cea807ecdef3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:21.279386 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1044/2072] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:55:21 2022] DELETE /identity/v3/projects/dce7a2fe13bf4130a256418df02eab16 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:21.282300 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5702f2af-c364-48e9-ba54-cea807ecdef3 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:21.282300 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-5702f2af-c364-48e9-ba54-cea807ecdef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:21.289510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45deac3e-7117-402f-b049-e15792f37e28 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:21.295161 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5702f2af-c364-48e9-ba54-cea807ecdef3 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:21.299188 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563120b2-60e0-45d4-99f7-7d0a73ed51e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:21.299908 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563120b2-60e0-45d4-99f7-7d0a73ed51e8 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:21.300081 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46389b3b-bf8f-4730-a39a-31ac821216ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:21.300449 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563120b2-60e0-45d4-99f7-7d0a73ed51e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:21.300813 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563120b2-60e0-45d4-99f7-7d0a73ed51e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:21.300926 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46389b3b-bf8f-4730-a39a-31ac821216ea admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:21.301252 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46389b3b-bf8f-4730-a39a-31ac821216ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:21.301592 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46389b3b-bf8f-4730-a39a-31ac821216ea admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:21.301702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563120b2-60e0-45d4-99f7-7d0a73ed51e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:21.302711 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46389b3b-bf8f-4730-a39a-31ac821216ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:55:21.303081 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46389b3b-bf8f-4730-a39a-31ac821216ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:21.303581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563120b2-60e0-45d4-99f7-7d0a73ed51e8 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:21.304865 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46389b3b-bf8f-4730-a39a-31ac821216ea admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:21.314088 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1029/2073] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:55:21.314950 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563120b2-60e0-45d4-99f7-7d0a73ed51e8 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:21.317210 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-563120b2-60e0-45d4-99f7-7d0a73ed51e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:21.320703 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1045/2074] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:55:21.320703 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46389b3b-bf8f-4730-a39a-31ac821216ea admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:21.320703 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-46389b3b-bf8f-4730-a39a-31ac821216ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:21.330275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563120b2-60e0-45d4-99f7-7d0a73ed51e8 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:21.333498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46389b3b-bf8f-4730-a39a-31ac821216ea admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:21.335814 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c99d3-ef0a-4c52-8232-400ea520ad35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:21.336551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c99d3-ef0a-4c52-8232-400ea520ad35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:21.337234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c99d3-ef0a-4c52-8232-400ea520ad35 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:21.337581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c99d3-ef0a-4c52-8232-400ea520ad35 admin admin] PATH_INFO: `/v3/users/abc598f96e2342eb9554055214468335` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:21.338027 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02604d12-69d6-4057-98b9-642abf614809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:21.338793 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02604d12-69d6-4057-98b9-642abf614809 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:21.339078 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02604d12-69d6-4057-98b9-642abf614809 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:21.339244 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5c99d3-ef0a-4c52-8232-400ea520ad35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abc598f96e2342eb9554055214468335)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:21.339345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02604d12-69d6-4057-98b9-642abf614809 admin admin] PATH_INFO: `/v3/projects/9bd31142621142e4b57095f5ad1c7019/users/42aeb986e27e4f1fa956ceabd15bea53/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:21.341310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5c99d3-ef0a-4c52-8232-400ea520ad35 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:21.354475 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02604d12-69d6-4057-98b9-642abf614809 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd31142621142e4b57095f5ad1c7019, user_id=42aeb986e27e4f1fa956ceabd15bea53, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:21.357195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02604d12-69d6-4057-98b9-642abf614809 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:21.423880 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1046/2075] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:55:21 2022] PUT /identity/v3/projects/9bd31142621142e4b57095f5ad1c7019/users/42aeb986e27e4f1fa956ceabd15bea53/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:21.425225 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02604d12-69d6-4057-98b9-642abf614809 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:21.484343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:21.489851 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02604d12-69d6-4057-98b9-642abf614809 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:21.494463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aabdd8f-0359-4fe1-99fc-fc1493822500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:21.495059 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabdd8f-0359-4fe1-99fc-fc1493822500 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:21.495337 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabdd8f-0359-4fe1-99fc-fc1493822500 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:21.495605 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabdd8f-0359-4fe1-99fc-fc1493822500 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:21.496219 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aabdd8f-0359-4fe1-99fc-fc1493822500 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:21.498158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aabdd8f-0359-4fe1-99fc-fc1493822500 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:21.581341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2c5c99d3-ef0a-4c52-8232-400ea520ad35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abc598f96e2342eb9554055214468335'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:21.593234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2c5c99d3-ef0a-4c52-8232-400ea520ad35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abc598f96e2342eb9554055214468335'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:21.654653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1047/2076] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:55:21 2022] POST /identity/v3/projects => generated 373 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:55:21.655929 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eed5a2-8be2-4acc-b5af-4ebd1043b3db None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:21.656213 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eed5a2-8be2-4acc-b5af-4ebd1043b3db None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:21.656558 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eed5a2-8be2-4acc-b5af-4ebd1043b3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:21.686449 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-63eed5a2-8be2-4acc-b5af-4ebd1043b3db None None] MFA Rules not processed for user `42aeb986e27e4f1fa956ceabd15bea53`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:21.687666 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-63eed5a2-8be2-4acc-b5af-4ebd1043b3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:21.688008 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2c5c99d3-ef0a-4c52-8232-400ea520ad35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abc598f96e2342eb9554055214468335'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:21.689034 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2c5c99d3-ef0a-4c52-8232-400ea520ad35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abc598f96e2342eb9554055214468335'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:21.689779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2c5c99d3-ef0a-4c52-8232-400ea520ad35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abc598f96e2342eb9554055214468335'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:21.705748 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1030/2077] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:55:21 2022] DELETE /identity/v3/users/abc598f96e2342eb9554055214468335 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:21.708767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c99d3-ef0a-4c52-8232-400ea520ad35 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:21.712985 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-2c5c99d3-ef0a-4c52-8232-400ea520ad35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:21.769897 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1048/2078] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:21.774341 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eed5a2-8be2-4acc-b5af-4ebd1043b3db None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:21.775760 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-63eed5a2-8be2-4acc-b5af-4ebd1043b3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:21.803636 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c99d3-ef0a-4c52-8232-400ea520ad35 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:21.810736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f53bd6b-19e3-4411-aebf-2bacde72cf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:21.811028 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53bd6b-19e3-4411-aebf-2bacde72cf90 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:21.811756 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53bd6b-19e3-4411-aebf-2bacde72cf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:21.812033 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53bd6b-19e3-4411-aebf-2bacde72cf90 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:21.812723 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f53bd6b-19e3-4411-aebf-2bacde72cf90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:21.814906 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f53bd6b-19e3-4411-aebf-2bacde72cf90 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:21.826140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:21.854696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eed5a2-8be2-4acc-b5af-4ebd1043b3db None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:21.859725 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a5cba-863f-40f8-ac25-cd3b214caac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:21.860348 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a5cba-863f-40f8-ac25-cd3b214caac6 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:21.860668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a5cba-863f-40f8-ac25-cd3b214caac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:21.860948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a5cba-863f-40f8-ac25-cd3b214caac6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:21.861580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a5cba-863f-40f8-ac25-cd3b214caac6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:21.864262 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a5cba-863f-40f8-ac25-cd3b214caac6 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:21.933349 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1031/2079] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:55:21 2022] POST /identity/v3/users => generated 287 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:55:21.937282 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f53bd6b-19e3-4411-aebf-2bacde72cf90 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:21.947323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f53bd6b-19e3-4411-aebf-2bacde72cf90 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:21.951875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4e951-97a3-41b3-81bf-9c460bca9cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:21.952453 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4e951-97a3-41b3-81bf-9c460bca9cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:21.952763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4e951-97a3-41b3-81bf-9c460bca9cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:21.953033 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4e951-97a3-41b3-81bf-9c460bca9cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:21.953662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4e951-97a3-41b3-81bf-9c460bca9cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:21.955320 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4e951-97a3-41b3-81bf-9c460bca9cc9 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:21.964524 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1049/2080] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:55:21 2022] POST /identity/v3/projects => generated 383 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:55:21.966132 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1032/2081] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 00:55:21.968992 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a5cba-863f-40f8-ac25-cd3b214caac6 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:21.969851 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4e951-97a3-41b3-81bf-9c460bca9cc9 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:21.979266 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4e951-97a3-41b3-81bf-9c460bca9cc9 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:22.032770 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:22.039127 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a5cba-863f-40f8-ac25-cd3b214caac6 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:22.044916 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8c530-67b2-40ea-87ac-c1f29b60bbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:22.045535 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8c530-67b2-40ea-87ac-c1f29b60bbce admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:22.045816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8c530-67b2-40ea-87ac-c1f29b60bbce admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:22.046085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8c530-67b2-40ea-87ac-c1f29b60bbce admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:22.046731 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a8c530-67b2-40ea-87ac-c1f29b60bbce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:22.048680 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a8c530-67b2-40ea-87ac-c1f29b60bbce admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:22.058505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db46bfd-027b-4e5d-bb46-25c52713e82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:22.059392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db46bfd-027b-4e5d-bb46-25c52713e82e admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:22.060177 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db46bfd-027b-4e5d-bb46-25c52713e82e admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:22.060528 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db46bfd-027b-4e5d-bb46-25c52713e82e admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:22.061182 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db46bfd-027b-4e5d-bb46-25c52713e82e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:22.062800 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db46bfd-027b-4e5d-bb46-25c52713e82e admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:22.073626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1033/2082] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 00:55:22.077939 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db46bfd-027b-4e5d-bb46-25c52713e82e admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:22.087920 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db46bfd-027b-4e5d-bb46-25c52713e82e admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:22.092791 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a0095-9271-446a-a6ed-620aeafe209b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:22.095369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a0095-9271-446a-a6ed-620aeafe209b admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:22.095369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a0095-9271-446a-a6ed-620aeafe209b admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:22.095369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a0095-9271-446a-a6ed-620aeafe209b admin admin] PATH_INFO: `/v3/projects/496f196120f7411ea12734fbf23167f3/users/ebc202cc3ba74039b9a93ba87f019a19/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:22.110149 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594a0095-9271-446a-a6ed-620aeafe209b admin admin] RBAC: Authorizing `identity:create_grant(project_id=496f196120f7411ea12734fbf23167f3, user_id=ebc202cc3ba74039b9a93ba87f019a19, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:22.113625 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594a0095-9271-446a-a6ed-620aeafe209b admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:22.150543 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1050/2083] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:55:21 2022] POST /identity/v3/users => generated 298 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:55:22.154429 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8c530-67b2-40ea-87ac-c1f29b60bbce admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:22.166808 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8c530-67b2-40ea-87ac-c1f29b60bbce admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:22.171510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36862e-c4df-4981-bb9b-e1c364fb2bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:22.172120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36862e-c4df-4981-bb9b-e1c364fb2bff admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:22.172398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36862e-c4df-4981-bb9b-e1c364fb2bff admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:22.172753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36862e-c4df-4981-bb9b-e1c364fb2bff admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:22.173424 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36862e-c4df-4981-bb9b-e1c364fb2bff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:22.174999 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36862e-c4df-4981-bb9b-e1c364fb2bff admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:22.187097 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1051/2084] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 00:55:22.192766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36862e-c4df-4981-bb9b-e1c364fb2bff admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:22.204096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36862e-c4df-4981-bb9b-e1c364fb2bff admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:22.209425 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1034/2085] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:55:22 2022] PUT /identity/v3/projects/496f196120f7411ea12734fbf23167f3/users/ebc202cc3ba74039b9a93ba87f019a19/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:55:22.225835 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393cb0f-3cac-4d94-88e6-338a86e2f477 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:22.226222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393cb0f-3cac-4d94-88e6-338a86e2f477 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:22.226373 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393cb0f-3cac-4d94-88e6-338a86e2f477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:22.258467 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-4393cb0f-3cac-4d94-88e6-338a86e2f477 None None] MFA Rules not processed for user `ebc202cc3ba74039b9a93ba87f019a19`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:22.260254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-4393cb0f-3cac-4d94-88e6-338a86e2f477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:22.269871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3818835-bc9c-4703-be95-14a54972958f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:22.270551 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3818835-bc9c-4703-be95-14a54972958f admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:22.270838 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3818835-bc9c-4703-be95-14a54972958f admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:22.271121 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3818835-bc9c-4703-be95-14a54972958f admin admin] PATH_INFO: `/v3/projects/df3021fb2a8d4111880c8f3d80d10333/users/a7aca84d1d3048c5aa0fd7546e09ef5b/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:22.291773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3818835-bc9c-4703-be95-14a54972958f admin admin] RBAC: Authorizing `identity:create_grant(project_id=df3021fb2a8d4111880c8f3d80d10333, user_id=a7aca84d1d3048c5aa0fd7546e09ef5b, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:22.294762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3818835-bc9c-4703-be95-14a54972958f admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:22.348425 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1035/2086] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:55:22.348425 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4393cb0f-3cac-4d94-88e6-338a86e2f477 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:22.348425 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-4393cb0f-3cac-4d94-88e6-338a86e2f477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:22.361842 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4393cb0f-3cac-4d94-88e6-338a86e2f477 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:22.369112 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1052/2087] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:55:22 2022] PUT /identity/v3/projects/df3021fb2a8d4111880c8f3d80d10333/users/a7aca84d1d3048c5aa0fd7546e09ef5b/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:55:22.370135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3818835-bc9c-4703-be95-14a54972958f admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:22.427897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:22.444778 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64184f26-76cc-4c87-8f80-a86a70945719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:22.445427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64184f26-76cc-4c87-8f80-a86a70945719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:22.445714 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64184f26-76cc-4c87-8f80-a86a70945719 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:22.446088 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64184f26-76cc-4c87-8f80-a86a70945719 admin admin] PATH_INFO: `/v3/projects/a629d286e21040728b5ec5165dc6bcff` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:22.447517 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64184f26-76cc-4c87-8f80-a86a70945719 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a629d286e21040728b5ec5165dc6bcff)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:22.449100 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64184f26-76cc-4c87-8f80-a86a70945719 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:22.452085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3818835-bc9c-4703-be95-14a54972958f admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:22.457712 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9362b2-7326-4bf8-b30f-4e69b3becc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:22.458325 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9362b2-7326-4bf8-b30f-4e69b3becc34 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:22.458633 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9362b2-7326-4bf8-b30f-4e69b3becc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:22.458866 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9362b2-7326-4bf8-b30f-4e69b3becc34 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:22.459490 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9362b2-7326-4bf8-b30f-4e69b3becc34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:22.461638 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9362b2-7326-4bf8-b30f-4e69b3becc34 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:22.761246 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-64184f26-76cc-4c87-8f80-a86a70945719 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a629d286e21040728b5ec5165dc6bcff', 'request_id': 'req-64184f26-76cc-4c87-8f80-a86a70945719'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:23.115927 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-64184f26-76cc-4c87-8f80-a86a70945719 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a629d286e21040728b5ec5165dc6bcff', 'request_id': 'req-64184f26-76cc-4c87-8f80-a86a70945719'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:23.126096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1053/2088] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:55:22 2022] POST /identity/v3/projects => generated 373 bytes in 753 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:55:23.127014 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-64184f26-76cc-4c87-8f80-a86a70945719 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a629d286e21040728b5ec5165dc6bcff', 'request_id': 'req-64184f26-76cc-4c87-8f80-a86a70945719'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:23.127139 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9362b2-7326-4bf8-b30f-4e69b3becc34 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:23.137263 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-64184f26-76cc-4c87-8f80-a86a70945719 admin admin] The token cache is being invalidate because project a629d286e21040728b5ec5165dc6bcff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:55:23.137610 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-64184f26-76cc-4c87-8f80-a86a70945719 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:23.138274 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-64184f26-76cc-4c87-8f80-a86a70945719 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:23.139974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1036/2089] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:55:22 2022] DELETE /identity/v3/projects/a629d286e21040728b5ec5165dc6bcff => generated 0 bytes in 794 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:23.143870 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64184f26-76cc-4c87-8f80-a86a70945719 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:23.145210 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-64184f26-76cc-4c87-8f80-a86a70945719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:23.172672 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:23.190401 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9362b2-7326-4bf8-b30f-4e69b3becc34 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:23.195527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e81369-a0bf-406c-a7ba-d833609c1bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:23.196136 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e81369-a0bf-406c-a7ba-d833609c1bee admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:23.196416 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e81369-a0bf-406c-a7ba-d833609c1bee admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:23.196718 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e81369-a0bf-406c-a7ba-d833609c1bee admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:23.197365 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e81369-a0bf-406c-a7ba-d833609c1bee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:23.198971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e81369-a0bf-406c-a7ba-d833609c1bee admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:23.209006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1054/2090] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:55:23.211731 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64184f26-76cc-4c87-8f80-a86a70945719 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:23.212832 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12571891-aaa1-442d-8256-7dba4f7db729 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:23.213123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12571891-aaa1-442d-8256-7dba4f7db729 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:23.213404 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12571891-aaa1-442d-8256-7dba4f7db729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:23.216969 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e0d55-66a8-4deb-a9ef-f1ed8c79af11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:23.217841 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e0d55-66a8-4deb-a9ef-f1ed8c79af11 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:23.218116 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e0d55-66a8-4deb-a9ef-f1ed8c79af11 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:23.218386 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e0d55-66a8-4deb-a9ef-f1ed8c79af11 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:23.218999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e0d55-66a8-4deb-a9ef-f1ed8c79af11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:23.220958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e0d55-66a8-4deb-a9ef-f1ed8c79af11 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:23.232967 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-12571891-aaa1-442d-8256-7dba4f7db729 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:23.234166 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-12571891-aaa1-442d-8256-7dba4f7db729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:23.248115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1055/2091] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:23.248115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12571891-aaa1-442d-8256-7dba4f7db729 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:23.248115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-12571891-aaa1-442d-8256-7dba4f7db729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:23.257059 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12571891-aaa1-442d-8256-7dba4f7db729 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:23.261590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6e2be-d539-472b-9029-cb53d8c9ca61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:23.262230 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6e2be-d539-472b-9029-cb53d8c9ca61 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:23.262512 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6e2be-d539-472b-9029-cb53d8c9ca61 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:23.262782 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6e2be-d539-472b-9029-cb53d8c9ca61 admin admin] PATH_INFO: `/v3/projects/df3021fb2a8d4111880c8f3d80d10333/users/a7aca84d1d3048c5aa0fd7546e09ef5b/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:23.264756 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6e2be-d539-472b-9029-cb53d8c9ca61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df3021fb2a8d4111880c8f3d80d10333, user_id=a7aca84d1d3048c5aa0fd7546e09ef5b, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:23.267315 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6e2be-d539-472b-9029-cb53d8c9ca61 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:23.277100 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1056/2092] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:55:23 2022] PUT /identity/v3/projects/df3021fb2a8d4111880c8f3d80d10333/users/a7aca84d1d3048c5aa0fd7546e09ef5b/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:23.278220 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04712a6-f0eb-4e06-8578-46a2ef84d54c None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:23.278506 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04712a6-f0eb-4e06-8578-46a2ef84d54c None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:23.278785 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04712a6-f0eb-4e06-8578-46a2ef84d54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:23.298292 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-d04712a6-f0eb-4e06-8578-46a2ef84d54c None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:23.302382 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-d04712a6-f0eb-4e06-8578-46a2ef84d54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:23.327276 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1037/2093] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:55:23 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:55:23.328599 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16115676-0fc4-4180-9a91-8469a29a2c33 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:23.334431 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16115676-0fc4-4180-9a91-8469a29a2c33 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:23.334431 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16115676-0fc4-4180-9a91-8469a29a2c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:23.358466 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-16115676-0fc4-4180-9a91-8469a29a2c33 None None] MFA Rules not processed for user `a7aca84d1d3048c5aa0fd7546e09ef5b`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:23.359697 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-16115676-0fc4-4180-9a91-8469a29a2c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:23.363970 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1057/2094] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:23.364860 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04712a6-f0eb-4e06-8578-46a2ef84d54c None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:23.375012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04712a6-f0eb-4e06-8578-46a2ef84d54c None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:23.379687 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4281fa9-1284-4ae4-bef2-554ac5b632c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:23.380281 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4281fa9-1284-4ae4-bef2-554ac5b632c1 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:23.380586 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4281fa9-1284-4ae4-bef2-554ac5b632c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:23.380858 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4281fa9-1284-4ae4-bef2-554ac5b632c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:23.381497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4281fa9-1284-4ae4-bef2-554ac5b632c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:23.383178 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4281fa9-1284-4ae4-bef2-554ac5b632c1 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:23.395937 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1058/2095] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:55:23.395937 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4281fa9-1284-4ae4-bef2-554ac5b632c1 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:23.406001 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4281fa9-1284-4ae4-bef2-554ac5b632c1 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:23.411365 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d00fb36-1262-4dec-961b-559677fc0e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:23.412399 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00fb36-1262-4dec-961b-559677fc0e13 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:23.412712 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00fb36-1262-4dec-961b-559677fc0e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:23.412988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00fb36-1262-4dec-961b-559677fc0e13 admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:23.413636 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d00fb36-1262-4dec-961b-559677fc0e13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:55:23.413941 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d00fb36-1262-4dec-961b-559677fc0e13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:23.415533 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d00fb36-1262-4dec-961b-559677fc0e13 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:23.428179 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:23.434141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1059/2096] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:55:23.434141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d00fb36-1262-4dec-961b-559677fc0e13 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:23.434388 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1038/2097] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:23 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:23.437752 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16115676-0fc4-4180-9a91-8469a29a2c33 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:23.445661 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d00fb36-1262-4dec-961b-559677fc0e13 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:23.447965 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16115676-0fc4-4180-9a91-8469a29a2c33 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:23.450418 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484f3a5-3c5f-4835-a1c8-e2507c1125b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:23.451085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484f3a5-3c5f-4835-a1c8-e2507c1125b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:23.451363 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484f3a5-3c5f-4835-a1c8-e2507c1125b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:23.451636 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484f3a5-3c5f-4835-a1c8-e2507c1125b0 admin admin] PATH_INFO: `/v3/projects/15bd36e42d924f9681cf9e87c6aa9afd/users/0bfa75a9e3734dfb852ace0db6939203/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:23.453026 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab680edc-0200-4311-aaab-835d90347810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:23.453712 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab680edc-0200-4311-aaab-835d90347810 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:23.454438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab680edc-0200-4311-aaab-835d90347810 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:23.454438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab680edc-0200-4311-aaab-835d90347810 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:23.455012 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab680edc-0200-4311-aaab-835d90347810 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:23.457069 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab680edc-0200-4311-aaab-835d90347810 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:23.466689 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484f3a5-3c5f-4835-a1c8-e2507c1125b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15bd36e42d924f9681cf9e87c6aa9afd, user_id=0bfa75a9e3734dfb852ace0db6939203, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:23.469382 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484f3a5-3c5f-4835-a1c8-e2507c1125b0 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:23.590499 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1060/2098] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:55:23 2022] PUT /identity/v3/projects/15bd36e42d924f9681cf9e87c6aa9afd/users/0bfa75a9e3734dfb852ace0db6939203/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:23.591417 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484f3a5-3c5f-4835-a1c8-e2507c1125b0 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:23.593202 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-d484f3a5-3c5f-4835-a1c8-e2507c1125b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:23.647822 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1039/2099] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:55:23 2022] POST /identity/v3/projects => generated 381 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:55:23.660607 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab680edc-0200-4311-aaab-835d90347810 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:23.660894 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484f3a5-3c5f-4835-a1c8-e2507c1125b0 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:23.704268 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102df85a-59e5-41a5-a0cb-ffb2dd2fad19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:23.704980 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102df85a-59e5-41a5-a0cb-ffb2dd2fad19 service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:23.705263 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102df85a-59e5-41a5-a0cb-ffb2dd2fad19 service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:23.705533 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102df85a-59e5-41a5-a0cb-ffb2dd2fad19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:23.707898 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-102df85a-59e5-41a5-a0cb-ffb2dd2fad19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:23.714277 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab680edc-0200-4311-aaab-835d90347810 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:23.719002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dbbe73-3eb6-43ee-b341-cd51d0c16dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:23.719647 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbbe73-3eb6-43ee-b341-cd51d0c16dad admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:23.719911 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbbe73-3eb6-43ee-b341-cd51d0c16dad admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:23.720191 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbbe73-3eb6-43ee-b341-cd51d0c16dad admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:23.720871 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dbbe73-3eb6-43ee-b341-cd51d0c16dad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:23.722530 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dbbe73-3eb6-43ee-b341-cd51d0c16dad admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:23.732219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1040/2100] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:55:23.734188 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dbbe73-3eb6-43ee-b341-cd51d0c16dad admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:23.744497 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dbbe73-3eb6-43ee-b341-cd51d0c16dad admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:23.749106 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc93b7-c52e-4cdf-baa8-1f35760ec470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:23.749700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc93b7-c52e-4cdf-baa8-1f35760ec470 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:23.753612 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc93b7-c52e-4cdf-baa8-1f35760ec470 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:23.753612 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc93b7-c52e-4cdf-baa8-1f35760ec470 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:23.753612 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc93b7-c52e-4cdf-baa8-1f35760ec470 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:23.753612 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc93b7-c52e-4cdf-baa8-1f35760ec470 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:23.768078 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102df85a-59e5-41a5-a0cb-ffb2dd2fad19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:23.769587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102df85a-59e5-41a5-a0cb-ffb2dd2fad19 service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:23.778629 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1061/2101] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:55:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:55:23.779415 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102df85a-59e5-41a5-a0cb-ffb2dd2fad19 service cinder] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:23.790257 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102df85a-59e5-41a5-a0cb-ffb2dd2fad19 service cinder] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:23.794946 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed39e62-5024-4b9a-bfa6-3ad95baf579a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:23.795588 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed39e62-5024-4b9a-bfa6-3ad95baf579a admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:23.795869 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed39e62-5024-4b9a-bfa6-3ad95baf579a admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:23.796145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed39e62-5024-4b9a-bfa6-3ad95baf579a admin admin] PATH_INFO: `/v3/projects/15bd36e42d924f9681cf9e87c6aa9afd/users/0bfa75a9e3734dfb852ace0db6939203/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:23.798089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed39e62-5024-4b9a-bfa6-3ad95baf579a admin admin] RBAC: Authorizing `identity:create_grant(project_id=15bd36e42d924f9681cf9e87c6aa9afd, user_id=0bfa75a9e3734dfb852ace0db6939203, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:23.802023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed39e62-5024-4b9a-bfa6-3ad95baf579a admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:23.812344 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1062/2102] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:55:23 2022] PUT /identity/v3/projects/15bd36e42d924f9681cf9e87c6aa9afd/users/0bfa75a9e3734dfb852ace0db6939203/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:23.825050 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fef7d-456e-4fea-a7eb-135913e7f89e None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:23.825339 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fef7d-456e-4fea-a7eb-135913e7f89e None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:23.825615 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fef7d-456e-4fea-a7eb-135913e7f89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:23.848684 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1041/2103] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:55:23 2022] POST /identity/v3/users => generated 291 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:55:23.853890 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc93b7-c52e-4cdf-baa8-1f35760ec470 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:23.861855 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-b40fef7d-456e-4fea-a7eb-135913e7f89e None None] MFA Rules not processed for user `0bfa75a9e3734dfb852ace0db6939203`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:23.864072 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-b40fef7d-456e-4fea-a7eb-135913e7f89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:23.865479 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc93b7-c52e-4cdf-baa8-1f35760ec470 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:23.902931 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:23.929316 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86a7e3-2626-43c8-9b08-d30635cadc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:23.930359 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86a7e3-2626-43c8-9b08-d30635cadc7c admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:23.930653 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86a7e3-2626-43c8-9b08-d30635cadc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:23.930935 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86a7e3-2626-43c8-9b08-d30635cadc7c admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:23.931686 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd86a7e3-2626-43c8-9b08-d30635cadc7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:23.933355 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd86a7e3-2626-43c8-9b08-d30635cadc7c admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:23.946338 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1042/2104] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 00:55:23.946507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1063/2105] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:55:23.947412 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86a7e3-2626-43c8-9b08-d30635cadc7c admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:23.956536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40fef7d-456e-4fea-a7eb-135913e7f89e None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:23.958516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86a7e3-2626-43c8-9b08-d30635cadc7c admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:23.958683 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-b40fef7d-456e-4fea-a7eb-135913e7f89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:23.962324 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839c6c7-9e5c-4d4e-9c5e-0756b0ced221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:23.962937 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839c6c7-9e5c-4d4e-9c5e-0756b0ced221 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:23.963205 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839c6c7-9e5c-4d4e-9c5e-0756b0ced221 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:23.963479 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839c6c7-9e5c-4d4e-9c5e-0756b0ced221 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:23.964116 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839c6c7-9e5c-4d4e-9c5e-0756b0ced221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:23.965728 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839c6c7-9e5c-4d4e-9c5e-0756b0ced221 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:23.976505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1043/2106] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:55:23.980713 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839c6c7-9e5c-4d4e-9c5e-0756b0ced221 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:23.991545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839c6c7-9e5c-4d4e-9c5e-0756b0ced221 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:23.996164 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d320b0-a91d-4dc3-87bb-11461457349d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:23.996834 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d320b0-a91d-4dc3-87bb-11461457349d admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:23.997400 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d320b0-a91d-4dc3-87bb-11461457349d admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:23.997682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d320b0-a91d-4dc3-87bb-11461457349d admin admin] PATH_INFO: `/v3/projects/e8cee98162a04de9af6dba3326d934ca/users/aca9f66ff8c54e5394f49050604e9de8/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:24.012551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d320b0-a91d-4dc3-87bb-11461457349d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8cee98162a04de9af6dba3326d934ca, user_id=aca9f66ff8c54e5394f49050604e9de8, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:24.015214 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d320b0-a91d-4dc3-87bb-11461457349d admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:24.017697 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40fef7d-456e-4fea-a7eb-135913e7f89e None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:24.021513 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949ddd1-f7be-4670-854e-901b2193c855 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:24.022108 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949ddd1-f7be-4670-854e-901b2193c855 service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:24.022428 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949ddd1-f7be-4670-854e-901b2193c855 service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:24.022704 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949ddd1-f7be-4670-854e-901b2193c855 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:24.031767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949ddd1-f7be-4670-854e-901b2193c855 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:24.033166 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949ddd1-f7be-4670-854e-901b2193c855 service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:24.043262 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1064/2107] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:55:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:55:24.069880 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1044/2108] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:55:23 2022] PUT /identity/v3/projects/e8cee98162a04de9af6dba3326d934ca/users/aca9f66ff8c54e5394f49050604e9de8/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:55:24.081318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ff3ca-ca05-430c-9c3c-1bd613bcc38c None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:24.081600 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ff3ca-ca05-430c-9c3c-1bd613bcc38c None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:24.081872 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ff3ca-ca05-430c-9c3c-1bd613bcc38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:24.113091 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-ff6ff3ca-ca05-430c-9c3c-1bd613bcc38c None None] MFA Rules not processed for user `aca9f66ff8c54e5394f49050604e9de8`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:24.114400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-ff6ff3ca-ca05-430c-9c3c-1bd613bcc38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:24.202798 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1065/2109] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:24.207461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d320b0-a91d-4dc3-87bb-11461457349d admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:24.281666 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d320b0-a91d-4dc3-87bb-11461457349d admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:24.288322 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0556ac-5448-4b30-893e-e1b2d27a22c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:24.288933 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0556ac-5448-4b30-893e-e1b2d27a22c4 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:24.289213 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0556ac-5448-4b30-893e-e1b2d27a22c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:24.289485 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0556ac-5448-4b30-893e-e1b2d27a22c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:24.290126 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0556ac-5448-4b30-893e-e1b2d27a22c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:24.292120 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0556ac-5448-4b30-893e-e1b2d27a22c4 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:24.399196 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1045/2110] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:55:24 2022] POST /identity/v3/projects => generated 381 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:55:24.402449 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ff3ca-ca05-430c-9c3c-1bd613bcc38c None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:24.415514 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ff3ca-ca05-430c-9c3c-1bd613bcc38c None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:24.500289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e6df4-8bef-43de-877e-a8d42dbb6abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:24.501255 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e6df4-8bef-43de-877e-a8d42dbb6abd admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:24.501736 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e6df4-8bef-43de-877e-a8d42dbb6abd admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:24.502095 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e6df4-8bef-43de-877e-a8d42dbb6abd admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:24.502859 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e6df4-8bef-43de-877e-a8d42dbb6abd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:24.505191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e6df4-8bef-43de-877e-a8d42dbb6abd admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:24.539353 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:24.603856 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1066/2111] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:55:24 2022] POST /identity/v3/users => generated 297 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:55:24.607764 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0556ac-5448-4b30-893e-e1b2d27a22c4 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:24.620547 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:24.624294 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0556ac-5448-4b30-893e-e1b2d27a22c4 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:24.629240 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ba963-67e7-4691-a62e-fcf6174921a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:24.629944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ba963-67e7-4691-a62e-fcf6174921a0 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:24.630877 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ba963-67e7-4691-a62e-fcf6174921a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:24.630877 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ba963-67e7-4691-a62e-fcf6174921a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:24.631280 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ba963-67e7-4691-a62e-fcf6174921a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:24.633018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ba963-67e7-4691-a62e-fcf6174921a0 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:24.644334 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1046/2112] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:55:24.648710 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e6df4-8bef-43de-877e-a8d42dbb6abd admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:24.658819 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e6df4-8bef-43de-877e-a8d42dbb6abd admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:24.671337 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ca1df-5a37-4007-ba1f-e83ef9fa0b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:24.671337 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ca1df-5a37-4007-ba1f-e83ef9fa0b4f admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:24.671337 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ca1df-5a37-4007-ba1f-e83ef9fa0b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:24.671337 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ca1df-5a37-4007-ba1f-e83ef9fa0b4f admin admin] PATH_INFO: `/v3/projects/4f06317949284dcc935e92131e094525/users/9ff4df38486f4944953d806535c802c2/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:24.681700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ca1df-5a37-4007-ba1f-e83ef9fa0b4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f06317949284dcc935e92131e094525, user_id=9ff4df38486f4944953d806535c802c2, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:24.684367 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ca1df-5a37-4007-ba1f-e83ef9fa0b4f admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:24.820033 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1067/2113] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:55:24 2022] PUT /identity/v3/projects/4f06317949284dcc935e92131e094525/users/9ff4df38486f4944953d806535c802c2/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:24.824217 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ba963-67e7-4691-a62e-fcf6174921a0 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:24.834967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ba963-67e7-4691-a62e-fcf6174921a0 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:24.898773 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91760556-255e-4200-b436-0a0a431e79c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:24.899395 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91760556-255e-4200-b436-0a0a431e79c2 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:24.899677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91760556-255e-4200-b436-0a0a431e79c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:24.899957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91760556-255e-4200-b436-0a0a431e79c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:24.900634 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91760556-255e-4200-b436-0a0a431e79c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:24.902232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91760556-255e-4200-b436-0a0a431e79c2 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:24.913489 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1047/2114] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 00:55:24.917870 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ca1df-5a37-4007-ba1f-e83ef9fa0b4f admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:24.929340 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ca1df-5a37-4007-ba1f-e83ef9fa0b4f admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:24.934390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ee2d5-83e8-494a-8721-f48d074fafd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:24.935059 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ee2d5-83e8-494a-8721-f48d074fafd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:24.935343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ee2d5-83e8-494a-8721-f48d074fafd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:24.935622 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ee2d5-83e8-494a-8721-f48d074fafd4 admin admin] PATH_INFO: `/v3/projects/4f06317949284dcc935e92131e094525/users/9ff4df38486f4944953d806535c802c2/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:24.937568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ee2d5-83e8-494a-8721-f48d074fafd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f06317949284dcc935e92131e094525, user_id=9ff4df38486f4944953d806535c802c2, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:24.940304 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ee2d5-83e8-494a-8721-f48d074fafd4 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:24.951014 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1068/2115] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:55:24 2022] PUT /identity/v3/projects/4f06317949284dcc935e92131e094525/users/9ff4df38486f4944953d806535c802c2/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:24.961310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3653e3-a091-4af1-b7d9-075d05c24b84 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:24.963637 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3653e3-a091-4af1-b7d9-075d05c24b84 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:24.963637 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3653e3-a091-4af1-b7d9-075d05c24b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:24.998626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-0e3653e3-a091-4af1-b7d9-075d05c24b84 None None] MFA Rules not processed for user `9ff4df38486f4944953d806535c802c2`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:24.999526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-0e3653e3-a091-4af1-b7d9-075d05c24b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:25.074035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1048/2116] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:24 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:25.085340 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ee2d5-83e8-494a-8721-f48d074fafd4 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:25.097679 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3653e3-a091-4af1-b7d9-075d05c24b84 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:25.142111 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ee2d5-83e8-494a-8721-f48d074fafd4 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:25.145896 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db0c81-ae0c-4626-92c0-240a38eb1115 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:25.146502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db0c81-ae0c-4626-92c0-240a38eb1115 service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:25.146797 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db0c81-ae0c-4626-92c0-240a38eb1115 service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:25.147059 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db0c81-ae0c-4626-92c0-240a38eb1115 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:25.159333 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db0c81-ae0c-4626-92c0-240a38eb1115 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:25.159333 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db0c81-ae0c-4626-92c0-240a38eb1115 service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:25.160487 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3653e3-a091-4af1-b7d9-075d05c24b84 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:25.165474 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4694c43-b53e-4e50-9475-e1886f670019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:25.166148 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4694c43-b53e-4e50-9475-e1886f670019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:25.166434 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4694c43-b53e-4e50-9475-e1886f670019 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:25.166707 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4694c43-b53e-4e50-9475-e1886f670019 admin admin] PATH_INFO: `/v3/users/ebc202cc3ba74039b9a93ba87f019a19` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:25.167853 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4694c43-b53e-4e50-9475-e1886f670019 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebc202cc3ba74039b9a93ba87f019a19)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:25.169072 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1069/2117] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:55:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:55:25.169820 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4694c43-b53e-4e50-9475-e1886f670019 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:25.274105 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db0c81-ae0c-4626-92c0-240a38eb1115 service cinder] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:25.286692 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db0c81-ae0c-4626-92c0-240a38eb1115 service cinder] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:25.287739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1377b-e23c-4999-9130-161e628a378f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:25.288373 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1377b-e23c-4999-9130-161e628a378f service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:25.288681 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1377b-e23c-4999-9130-161e628a378f service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:25.288958 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1377b-e23c-4999-9130-161e628a378f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:25.347444 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-b4694c43-b53e-4e50-9475-e1886f670019 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebc202cc3ba74039b9a93ba87f019a19'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:25.348689 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1377b-e23c-4999-9130-161e628a378f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:25.350129 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1377b-e23c-4999-9130-161e628a378f service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:25.360467 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1070/2118] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:55:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:55:25.363264 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:25.364256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-b4694c43-b53e-4e50-9475-e1886f670019 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebc202cc3ba74039b9a93ba87f019a19'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:25.476441 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-b4694c43-b53e-4e50-9475-e1886f670019 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebc202cc3ba74039b9a93ba87f019a19'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:25.478220 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-b4694c43-b53e-4e50-9475-e1886f670019 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebc202cc3ba74039b9a93ba87f019a19'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:25.479020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-b4694c43-b53e-4e50-9475-e1886f670019 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebc202cc3ba74039b9a93ba87f019a19'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:25.493748 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1049/2119] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:55:25 2022] DELETE /identity/v3/users/ebc202cc3ba74039b9a93ba87f019a19 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:55:25.498309 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c51d9e-9b93-4d67-a44b-c5722339b532 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:25.498588 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c51d9e-9b93-4d67-a44b-c5722339b532 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:25.498867 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c51d9e-9b93-4d67-a44b-c5722339b532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:25.510672 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:25.520583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-d2c51d9e-9b93-4d67-a44b-c5722339b532 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:25.526376 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-d2c51d9e-9b93-4d67-a44b-c5722339b532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:25.589746 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1071/2120] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:25.598646 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4694c43-b53e-4e50-9475-e1886f670019 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:25.600116 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-b4694c43-b53e-4e50-9475-e1886f670019 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:25.660292 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4694c43-b53e-4e50-9475-e1886f670019 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:25.664241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe650cbc-664d-4a2f-971a-70851aabdf21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:25.664899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe650cbc-664d-4a2f-971a-70851aabdf21 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:25.665175 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe650cbc-664d-4a2f-971a-70851aabdf21 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:25.665451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe650cbc-664d-4a2f-971a-70851aabdf21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:25.674291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe650cbc-664d-4a2f-971a-70851aabdf21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:25.676416 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe650cbc-664d-4a2f-971a-70851aabdf21 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:25.685874 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1050/2121] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:55:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:55:25.830345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c51d9e-9b93-4d67-a44b-c5722339b532 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:25.831732 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-d2c51d9e-9b93-4d67-a44b-c5722339b532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:25.901569 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c51d9e-9b93-4d67-a44b-c5722339b532 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:25.911071 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f3353-efc0-4ffe-88bd-8c36e6feef46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:25.912096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f3353-efc0-4ffe-88bd-8c36e6feef46 service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:25.912375 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f3353-efc0-4ffe-88bd-8c36e6feef46 service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:25.912725 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f3353-efc0-4ffe-88bd-8c36e6feef46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:25.914579 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-d92f3353-efc0-4ffe-88bd-8c36e6feef46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:25.976763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f3353-efc0-4ffe-88bd-8c36e6feef46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:25.978469 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f3353-efc0-4ffe-88bd-8c36e6feef46 service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:25.987654 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1072/2122] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:55:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:55:26.204710 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe650cbc-664d-4a2f-971a-70851aabdf21 service neutron] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:26.206683 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-fe650cbc-664d-4a2f-971a-70851aabdf21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:26.237931 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe650cbc-664d-4a2f-971a-70851aabdf21 service neutron] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:26.246538 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4b350-e539-4216-8cb1-104bd691cddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:26.247918 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4b350-e539-4216-8cb1-104bd691cddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:26.248526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4b350-e539-4216-8cb1-104bd691cddd admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:26.248806 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4b350-e539-4216-8cb1-104bd691cddd admin admin] PATH_INFO: `/v3/projects/496f196120f7411ea12734fbf23167f3` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:26.251589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4b350-e539-4216-8cb1-104bd691cddd admin admin] RBAC: Authorizing `identity:delete_project(project_id=496f196120f7411ea12734fbf23167f3)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:26.254699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4b350-e539-4216-8cb1-104bd691cddd admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:26.279389 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:26.369367 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:26.380611 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f9c4b350-e539-4216-8cb1-104bd691cddd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '496f196120f7411ea12734fbf23167f3', 'request_id': 'req-f9c4b350-e539-4216-8cb1-104bd691cddd'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:26.441574 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f9c4b350-e539-4216-8cb1-104bd691cddd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '496f196120f7411ea12734fbf23167f3', 'request_id': 'req-f9c4b350-e539-4216-8cb1-104bd691cddd'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:26.448572 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f9c4b350-e539-4216-8cb1-104bd691cddd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '496f196120f7411ea12734fbf23167f3', 'request_id': 'req-f9c4b350-e539-4216-8cb1-104bd691cddd'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:26.460370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f9c4b350-e539-4216-8cb1-104bd691cddd admin admin] The token cache is being invalidate because project 496f196120f7411ea12734fbf23167f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:55:26.460839 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f9c4b350-e539-4216-8cb1-104bd691cddd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:26.464782 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f9c4b350-e539-4216-8cb1-104bd691cddd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:26.469804 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1051/2123] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:55:26 2022] DELETE /identity/v3/projects/496f196120f7411ea12734fbf23167f3 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:26.473761 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f3353-efc0-4ffe-88bd-8c36e6feef46 service cinder] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:26.485698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f3353-efc0-4ffe-88bd-8c36e6feef46 service cinder] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:26.545244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20460765-5b93-47a1-8211-6d9f5cce770b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:26.545782 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20460765-5b93-47a1-8211-6d9f5cce770b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:26.546063 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20460765-5b93-47a1-8211-6d9f5cce770b admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:26.546357 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20460765-5b93-47a1-8211-6d9f5cce770b admin admin] PATH_INFO: `/v3/users/0bfa75a9e3734dfb852ace0db6939203` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:26.547512 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20460765-5b93-47a1-8211-6d9f5cce770b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bfa75a9e3734dfb852ace0db6939203)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:26.549504 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20460765-5b93-47a1-8211-6d9f5cce770b admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:26.723106 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-20460765-5b93-47a1-8211-6d9f5cce770b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bfa75a9e3734dfb852ace0db6939203'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:26.723713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-20460765-5b93-47a1-8211-6d9f5cce770b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bfa75a9e3734dfb852ace0db6939203'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:26.735301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-20460765-5b93-47a1-8211-6d9f5cce770b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bfa75a9e3734dfb852ace0db6939203'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:26.835937 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-20460765-5b93-47a1-8211-6d9f5cce770b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bfa75a9e3734dfb852ace0db6939203'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:26.836592 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-20460765-5b93-47a1-8211-6d9f5cce770b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bfa75a9e3734dfb852ace0db6939203'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:26.850376 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1073/2124] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:55:26 2022] DELETE /identity/v3/users/0bfa75a9e3734dfb852ace0db6939203 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:27.347276 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4b350-e539-4216-8cb1-104bd691cddd admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:27.349331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-f9c4b350-e539-4216-8cb1-104bd691cddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:27.372823 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4b350-e539-4216-8cb1-104bd691cddd admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:27.455642 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20460765-5b93-47a1-8211-6d9f5cce770b admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:27.457844 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-20460765-5b93-47a1-8211-6d9f5cce770b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:27.463770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175b4ac-ce0a-4c8e-801f-de6a7923d9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:27.464413 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175b4ac-ce0a-4c8e-801f-de6a7923d9fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:27.464764 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175b4ac-ce0a-4c8e-801f-de6a7923d9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:27.465046 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175b4ac-ce0a-4c8e-801f-de6a7923d9fb admin admin] PATH_INFO: `/v3/projects/15bd36e42d924f9681cf9e87c6aa9afd` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:27.466188 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0175b4ac-ce0a-4c8e-801f-de6a7923d9fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=15bd36e42d924f9681cf9e87c6aa9afd)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:27.468191 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0175b4ac-ce0a-4c8e-801f-de6a7923d9fb admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:27.516680 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20460765-5b93-47a1-8211-6d9f5cce770b admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:27.521388 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da366cdb-bfe9-4270-acbc-a421ae8bc95f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:27.522069 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da366cdb-bfe9-4270-acbc-a421ae8bc95f service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:27.522381 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da366cdb-bfe9-4270-acbc-a421ae8bc95f service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:27.522687 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da366cdb-bfe9-4270-acbc-a421ae8bc95f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:27.524615 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-da366cdb-bfe9-4270-acbc-a421ae8bc95f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:27.530686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:27.578330 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0175b4ac-ce0a-4c8e-801f-de6a7923d9fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15bd36e42d924f9681cf9e87c6aa9afd', 'request_id': 'req-0175b4ac-ce0a-4c8e-801f-de6a7923d9fb'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:27.592257 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da366cdb-bfe9-4270-acbc-a421ae8bc95f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:27.593880 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da366cdb-bfe9-4270-acbc-a421ae8bc95f service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:27.604334 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1074/2125] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:55:27 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:55:27.681155 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0175b4ac-ce0a-4c8e-801f-de6a7923d9fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15bd36e42d924f9681cf9e87c6aa9afd', 'request_id': 'req-0175b4ac-ce0a-4c8e-801f-de6a7923d9fb'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:27.688677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0175b4ac-ce0a-4c8e-801f-de6a7923d9fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15bd36e42d924f9681cf9e87c6aa9afd', 'request_id': 'req-0175b4ac-ce0a-4c8e-801f-de6a7923d9fb'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:27.699902 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0175b4ac-ce0a-4c8e-801f-de6a7923d9fb admin admin] The token cache is being invalidate because project 15bd36e42d924f9681cf9e87c6aa9afd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:55:27.700191 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0175b4ac-ce0a-4c8e-801f-de6a7923d9fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:27.700865 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0175b4ac-ce0a-4c8e-801f-de6a7923d9fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:27.702387 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1052/2126] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:55:27 2022] DELETE /identity/v3/projects/15bd36e42d924f9681cf9e87c6aa9afd => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:27.707963 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1dd5b8-f4c3-4ab4-b28d-4af7a8b18c70 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:27.708269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1dd5b8-f4c3-4ab4-b28d-4af7a8b18c70 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:27.708577 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1dd5b8-f4c3-4ab4-b28d-4af7a8b18c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:27.730180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-1a1dd5b8-f4c3-4ab4-b28d-4af7a8b18c70 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:27.731407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-1a1dd5b8-f4c3-4ab4-b28d-4af7a8b18c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:27.801806 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1075/2127] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:27.808876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444f188-e499-4f42-86b2-596db9bdff3d None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:27.809167 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444f188-e499-4f42-86b2-596db9bdff3d None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:27.809446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444f188-e499-4f42-86b2-596db9bdff3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:27.830172 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-7444f188-e499-4f42-86b2-596db9bdff3d None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:27.831985 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-7444f188-e499-4f42-86b2-596db9bdff3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:27.840455 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1053/2128] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:27.848732 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1dd5b8-f4c3-4ab4-b28d-4af7a8b18c70 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:27.856428 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1dd5b8-f4c3-4ab4-b28d-4af7a8b18c70 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:27.861109 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b32822-d972-4922-a942-460e9a292b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:27.861714 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b32822-d972-4922-a942-460e9a292b6d admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:27.861992 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b32822-d972-4922-a942-460e9a292b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:27.862263 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b32822-d972-4922-a942-460e9a292b6d admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:27.862947 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b32822-d972-4922-a942-460e9a292b6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:55:27.863261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b32822-d972-4922-a942-460e9a292b6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:27.864911 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b32822-d972-4922-a942-460e9a292b6d admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:27.877132 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1076/2129] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:55:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:55:27.882780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444f188-e499-4f42-86b2-596db9bdff3d None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:27.896778 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444f188-e499-4f42-86b2-596db9bdff3d None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:27.901812 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815cbb0-70cc-447d-a1bf-6efcfd31d45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:27.902446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815cbb0-70cc-447d-a1bf-6efcfd31d45b admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:27.908267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815cbb0-70cc-447d-a1bf-6efcfd31d45b admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:27.908267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815cbb0-70cc-447d-a1bf-6efcfd31d45b admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:27.908267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b815cbb0-70cc-447d-a1bf-6efcfd31d45b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:27.908267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b815cbb0-70cc-447d-a1bf-6efcfd31d45b admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:28.009031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1054/2130] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:55:27 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:55:28.013462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b32822-d972-4922-a942-460e9a292b6d admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:28.023996 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b32822-d972-4922-a942-460e9a292b6d admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:28.083114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc50b0b-5e13-4390-a6c3-d77b152bfba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:28.083644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc50b0b-5e13-4390-a6c3-d77b152bfba7 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:28.083929 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc50b0b-5e13-4390-a6c3-d77b152bfba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:28.084204 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc50b0b-5e13-4390-a6c3-d77b152bfba7 admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:28.084880 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc50b0b-5e13-4390-a6c3-d77b152bfba7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:28.086797 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc50b0b-5e13-4390-a6c3-d77b152bfba7 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:28.183983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1077/2131] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:55:28 2022] POST /identity/v3/users => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:55:28.188290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815cbb0-70cc-447d-a1bf-6efcfd31d45b admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:28.199133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815cbb0-70cc-447d-a1bf-6efcfd31d45b admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:28.204292 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40a198-be6c-4fe5-94d3-e3e740022a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:28.212389 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40a198-be6c-4fe5-94d3-e3e740022a9a admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:28.212389 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40a198-be6c-4fe5-94d3-e3e740022a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:28.212389 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40a198-be6c-4fe5-94d3-e3e740022a9a admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:28.212389 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40a198-be6c-4fe5-94d3-e3e740022a9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:28.212389 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40a198-be6c-4fe5-94d3-e3e740022a9a admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:28.212389 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:28.225156 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1055/2132] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:55:28.229318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc50b0b-5e13-4390-a6c3-d77b152bfba7 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:28.239786 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc50b0b-5e13-4390-a6c3-d77b152bfba7 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:28.244493 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d633f9e8-fe1c-4cd5-afb4-d90f81b16c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:28.245136 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633f9e8-fe1c-4cd5-afb4-d90f81b16c92 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:28.246545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633f9e8-fe1c-4cd5-afb4-d90f81b16c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:28.246545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633f9e8-fe1c-4cd5-afb4-d90f81b16c92 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:28.246545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d633f9e8-fe1c-4cd5-afb4-d90f81b16c92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:28.247927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d633f9e8-fe1c-4cd5-afb4-d90f81b16c92 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:28.258781 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1078/2133] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:55:28.263063 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40a198-be6c-4fe5-94d3-e3e740022a9a admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:28.274161 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40a198-be6c-4fe5-94d3-e3e740022a9a admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:28.278840 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f85860-60ef-483b-be84-8beaf5e2dd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:28.286977 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f85860-60ef-483b-be84-8beaf5e2dd34 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:28.286977 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f85860-60ef-483b-be84-8beaf5e2dd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:28.286977 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f85860-60ef-483b-be84-8beaf5e2dd34 admin admin] PATH_INFO: `/v3/projects/7b16fe486c914268b647025b18c7f437/users/56765a3804124aa183444b26962e6299/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:28.295404 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f85860-60ef-483b-be84-8beaf5e2dd34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b16fe486c914268b647025b18c7f437, user_id=56765a3804124aa183444b26962e6299, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:28.298181 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f85860-60ef-483b-be84-8beaf5e2dd34 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:28.350336 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1056/2134] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:55:28 2022] PUT /identity/v3/projects/7b16fe486c914268b647025b18c7f437/users/56765a3804124aa183444b26962e6299/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:28.360810 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268b450-0f21-4276-9b24-6200dc1b7046 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:28.361100 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268b450-0f21-4276-9b24-6200dc1b7046 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:28.361380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268b450-0f21-4276-9b24-6200dc1b7046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:28.384025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:28.394249 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-f268b450-0f21-4276-9b24-6200dc1b7046 None None] MFA Rules not processed for user `56765a3804124aa183444b26962e6299`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:28.395570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f268b450-0f21-4276-9b24-6200dc1b7046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:28.473211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1079/2135] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:28.477862 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f85860-60ef-483b-be84-8beaf5e2dd34 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:28.547132 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f85860-60ef-483b-be84-8beaf5e2dd34 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:28.552376 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1337c-e1c9-4541-bff3-08f161f4cee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:28.553045 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1337c-e1c9-4541-bff3-08f161f4cee7 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:28.553344 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1337c-e1c9-4541-bff3-08f161f4cee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:28.553618 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1337c-e1c9-4541-bff3-08f161f4cee7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:28.554240 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1337c-e1c9-4541-bff3-08f161f4cee7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:28.556179 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1337c-e1c9-4541-bff3-08f161f4cee7 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:28.672499 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268b450-0f21-4276-9b24-6200dc1b7046 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:28.673922 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f268b450-0f21-4276-9b24-6200dc1b7046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:28.687950 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268b450-0f21-4276-9b24-6200dc1b7046 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:28.692760 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696efb93-ff07-47d4-a9df-35dc52fce695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:28.693388 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696efb93-ff07-47d4-a9df-35dc52fce695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:28.693673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696efb93-ff07-47d4-a9df-35dc52fce695 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:28.693948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696efb93-ff07-47d4-a9df-35dc52fce695 admin admin] PATH_INFO: `/v3/users/aca9f66ff8c54e5394f49050604e9de8` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:28.695096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696efb93-ff07-47d4-a9df-35dc52fce695 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aca9f66ff8c54e5394f49050604e9de8)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:28.697041 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696efb93-ff07-47d4-a9df-35dc52fce695 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:28.729273 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1057/2136] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:55:28 2022] POST /identity/v3/projects => generated 375 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:55:28.733425 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1337c-e1c9-4541-bff3-08f161f4cee7 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:28.795852 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1337c-e1c9-4541-bff3-08f161f4cee7 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:28.801257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214526b-cd7c-45af-8b60-f420bfcb0731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:28.801951 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214526b-cd7c-45af-8b60-f420bfcb0731 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:28.802233 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214526b-cd7c-45af-8b60-f420bfcb0731 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:28.802527 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214526b-cd7c-45af-8b60-f420bfcb0731 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:28.803155 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6214526b-cd7c-45af-8b60-f420bfcb0731 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:28.805205 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6214526b-cd7c-45af-8b60-f420bfcb0731 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:28.915824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1058/2137] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:55:28 2022] POST /identity/v3/users => generated 294 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:55:28.919464 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214526b-cd7c-45af-8b60-f420bfcb0731 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:28.924035 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-696efb93-ff07-47d4-a9df-35dc52fce695 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aca9f66ff8c54e5394f49050604e9de8'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:28.928356 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-696efb93-ff07-47d4-a9df-35dc52fce695 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aca9f66ff8c54e5394f49050604e9de8'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:28.929578 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214526b-cd7c-45af-8b60-f420bfcb0731 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:28.935927 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e5f224-82bd-4d97-9a94-d9c17d77b47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:28.936634 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5f224-82bd-4d97-9a94-d9c17d77b47d admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:28.937239 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5f224-82bd-4d97-9a94-d9c17d77b47d admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:28.937239 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5f224-82bd-4d97-9a94-d9c17d77b47d admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:28.937991 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e5f224-82bd-4d97-9a94-d9c17d77b47d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:28.938694 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-696efb93-ff07-47d4-a9df-35dc52fce695 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aca9f66ff8c54e5394f49050604e9de8'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:28.942415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e5f224-82bd-4d97-9a94-d9c17d77b47d admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:28.953187 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1059/2138] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 00:55:28.957925 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e5f224-82bd-4d97-9a94-d9c17d77b47d admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:28.959912 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-f8e5f224-82bd-4d97-9a94-d9c17d77b47d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:28.977816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e5f224-82bd-4d97-9a94-d9c17d77b47d admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:28.982576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88252f35-4495-401f-8a53-44669d4e8e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:28.983236 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88252f35-4495-401f-8a53-44669d4e8e4f admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:28.983516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88252f35-4495-401f-8a53-44669d4e8e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:28.983789 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88252f35-4495-401f-8a53-44669d4e8e4f admin admin] PATH_INFO: `/v3/projects/5d131a9aea474f21b0fce6c1729670bb/users/5d289425b15d4d4c87de65d34c12273b/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:29.016762 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88252f35-4495-401f-8a53-44669d4e8e4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d131a9aea474f21b0fce6c1729670bb, user_id=5d289425b15d4d4c87de65d34c12273b, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:29.016762 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88252f35-4495-401f-8a53-44669d4e8e4f admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:29.019233 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-696efb93-ff07-47d4-a9df-35dc52fce695 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aca9f66ff8c54e5394f49050604e9de8'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:29.019930 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-696efb93-ff07-47d4-a9df-35dc52fce695 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aca9f66ff8c54e5394f49050604e9de8'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:29.041895 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1080/2139] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:55:28 2022] DELETE /identity/v3/users/aca9f66ff8c54e5394f49050604e9de8 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:55:29.043355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a995ac-d25f-4245-a934-9bbccceeb5b3 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:29.043780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a995ac-d25f-4245-a934-9bbccceeb5b3 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:29.044329 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a995ac-d25f-4245-a934-9bbccceeb5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:29.068915 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-c0a995ac-d25f-4245-a934-9bbccceeb5b3 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:29.070153 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-c0a995ac-d25f-4245-a934-9bbccceeb5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:29.115502 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1060/2140] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:55:28 2022] PUT /identity/v3/projects/5d131a9aea474f21b0fce6c1729670bb/users/5d289425b15d4d4c87de65d34c12273b/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:55:29.121695 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88252f35-4495-401f-8a53-44669d4e8e4f admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:29.135916 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:29.146973 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1081/2141] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:55:29.158895 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a995ac-d25f-4245-a934-9bbccceeb5b3 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:29.160303 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-c0a995ac-d25f-4245-a934-9bbccceeb5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:29.180188 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:29.190951 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88252f35-4495-401f-8a53-44669d4e8e4f admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:29.195866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30056ef0-8139-4518-9f3a-db1bfdee4800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:29.196467 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30056ef0-8139-4518-9f3a-db1bfdee4800 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:29.196770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30056ef0-8139-4518-9f3a-db1bfdee4800 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:29.197049 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30056ef0-8139-4518-9f3a-db1bfdee4800 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:29.197693 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30056ef0-8139-4518-9f3a-db1bfdee4800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:29.200597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30056ef0-8139-4518-9f3a-db1bfdee4800 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:29.224503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1061/2142] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:55:29.228687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30056ef0-8139-4518-9f3a-db1bfdee4800 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:29.231645 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a995ac-d25f-4245-a934-9bbccceeb5b3 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:29.240104 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30056ef0-8139-4518-9f3a-db1bfdee4800 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:29.244949 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3fd14-2a7a-4c86-95db-209b16954f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:29.245606 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3fd14-2a7a-4c86-95db-209b16954f5e admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:29.245882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3fd14-2a7a-4c86-95db-209b16954f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:29.246161 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3fd14-2a7a-4c86-95db-209b16954f5e admin admin] PATH_INFO: `/v3/projects/5d131a9aea474f21b0fce6c1729670bb/users/5d289425b15d4d4c87de65d34c12273b/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:29.248175 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef3fd14-2a7a-4c86-95db-209b16954f5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d131a9aea474f21b0fce6c1729670bb, user_id=5d289425b15d4d4c87de65d34c12273b, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:29.251045 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef3fd14-2a7a-4c86-95db-209b16954f5e admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:29.266119 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1062/2143] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:55:29 2022] PUT /identity/v3/projects/5d131a9aea474f21b0fce6c1729670bb/users/5d289425b15d4d4c87de65d34c12273b/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:55:29.277980 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7b41d-5730-4584-9b60-b188fb567b68 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:29.278531 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7b41d-5730-4584-9b60-b188fb567b68 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:29.279001 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7b41d-5730-4584-9b60-b188fb567b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:29.290734 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a954d-61e1-4fa9-8955-95633a141350 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:29.291413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a954d-61e1-4fa9-8955-95633a141350 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:29.291718 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a954d-61e1-4fa9-8955-95633a141350 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:29.292135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a954d-61e1-4fa9-8955-95633a141350 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:29.311753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a954d-61e1-4fa9-8955-95633a141350 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:29.313733 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a954d-61e1-4fa9-8955-95633a141350 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:29.322831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-2bf7b41d-5730-4584-9b60-b188fb567b68 None None] MFA Rules not processed for user `5d289425b15d4d4c87de65d34c12273b`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:29.324559 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-2bf7b41d-5730-4584-9b60-b188fb567b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:29.324706 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1082/2144] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:55:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:55:29.395998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1063/2145] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:29 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:55:29.423787 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a954d-61e1-4fa9-8955-95633a141350 service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:29.425419 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a76a954d-61e1-4fa9-8955-95633a141350 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:29.490154 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a954d-61e1-4fa9-8955-95633a141350 service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:29.557015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab117c58-e0a4-4b09-a6dd-f69134c6a8fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:29.558624 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab117c58-e0a4-4b09-a6dd-f69134c6a8fa service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:29.558624 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab117c58-e0a4-4b09-a6dd-f69134c6a8fa service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:29.558833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab117c58-e0a4-4b09-a6dd-f69134c6a8fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:29.561951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-ab117c58-e0a4-4b09-a6dd-f69134c6a8fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:29.636815 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:29.665196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab117c58-e0a4-4b09-a6dd-f69134c6a8fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:29.666710 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab117c58-e0a4-4b09-a6dd-f69134c6a8fa service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:29.688313 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1083/2146] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:55:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:55:29.855451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7b41d-5730-4584-9b60-b188fb567b68 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:29.856950 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-2bf7b41d-5730-4584-9b60-b188fb567b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:29.929880 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7b41d-5730-4584-9b60-b188fb567b68 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:29.934361 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d509053-5715-4196-9707-7f0a92a9ece7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:29.934989 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d509053-5715-4196-9707-7f0a92a9ece7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:29.935312 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d509053-5715-4196-9707-7f0a92a9ece7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:29.935545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d509053-5715-4196-9707-7f0a92a9ece7 admin admin] PATH_INFO: `/v3/projects/e8cee98162a04de9af6dba3326d934ca` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:29.936678 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d509053-5715-4196-9707-7f0a92a9ece7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8cee98162a04de9af6dba3326d934ca)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:29.938592 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d509053-5715-4196-9707-7f0a92a9ece7 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:29.963576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:30.019328 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2d509053-5715-4196-9707-7f0a92a9ece7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8cee98162a04de9af6dba3326d934ca', 'request_id': 'req-2d509053-5715-4196-9707-7f0a92a9ece7'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:30.082568 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2d509053-5715-4196-9707-7f0a92a9ece7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8cee98162a04de9af6dba3326d934ca', 'request_id': 'req-2d509053-5715-4196-9707-7f0a92a9ece7'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:30.090246 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2d509053-5715-4196-9707-7f0a92a9ece7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8cee98162a04de9af6dba3326d934ca', 'request_id': 'req-2d509053-5715-4196-9707-7f0a92a9ece7'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:30.100626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2d509053-5715-4196-9707-7f0a92a9ece7 admin admin] The token cache is being invalidate because project e8cee98162a04de9af6dba3326d934ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:55:30.100921 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2d509053-5715-4196-9707-7f0a92a9ece7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:30.101584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2d509053-5715-4196-9707-7f0a92a9ece7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:30.103172 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1064/2147] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:55:29 2022] DELETE /identity/v3/projects/e8cee98162a04de9af6dba3326d934ca => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:30.107319 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab117c58-e0a4-4b09-a6dd-f69134c6a8fa service cinder] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:30.118043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab117c58-e0a4-4b09-a6dd-f69134c6a8fa service cinder] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:30.175101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb857db6-6b53-44f2-85a3-0329a35bf753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:30.175722 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb857db6-6b53-44f2-85a3-0329a35bf753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:30.175997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb857db6-6b53-44f2-85a3-0329a35bf753 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:30.176265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb857db6-6b53-44f2-85a3-0329a35bf753 admin admin] PATH_INFO: `/v3/users/9ff4df38486f4944953d806535c802c2` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:30.177392 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb857db6-6b53-44f2-85a3-0329a35bf753 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ff4df38486f4944953d806535c802c2)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:30.179332 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb857db6-6b53-44f2-85a3-0329a35bf753 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:30.352604 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-eb857db6-6b53-44f2-85a3-0329a35bf753 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ff4df38486f4944953d806535c802c2'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:30.353274 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-eb857db6-6b53-44f2-85a3-0329a35bf753 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ff4df38486f4944953d806535c802c2'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:30.366232 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-eb857db6-6b53-44f2-85a3-0329a35bf753 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ff4df38486f4944953d806535c802c2'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:30.446930 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-eb857db6-6b53-44f2-85a3-0329a35bf753 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ff4df38486f4944953d806535c802c2'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:30.447575 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-eb857db6-6b53-44f2-85a3-0329a35bf753 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ff4df38486f4944953d806535c802c2'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:30.460369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1084/2148] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:55:30 2022] DELETE /identity/v3/users/9ff4df38486f4944953d806535c802c2 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:31.037254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d509053-5715-4196-9707-7f0a92a9ece7 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:31.039385 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-2d509053-5715-4196-9707-7f0a92a9ece7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:31.054079 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d509053-5715-4196-9707-7f0a92a9ece7 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:31.115771 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd2b3b-42c0-4922-b8e2-3a2a371b1966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:31.116407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd2b3b-42c0-4922-b8e2-3a2a371b1966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:31.116719 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd2b3b-42c0-4922-b8e2-3a2a371b1966 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:31.116996 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd2b3b-42c0-4922-b8e2-3a2a371b1966 admin admin] PATH_INFO: `/v3/projects/4f06317949284dcc935e92131e094525` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:31.118114 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd2b3b-42c0-4922-b8e2-3a2a371b1966 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f06317949284dcc935e92131e094525)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:31.120078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd2b3b-42c0-4922-b8e2-3a2a371b1966 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:31.257442 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-d5bd2b3b-42c0-4922-b8e2-3a2a371b1966 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f06317949284dcc935e92131e094525', 'request_id': 'req-d5bd2b3b-42c0-4922-b8e2-3a2a371b1966'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:31.344457 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-d5bd2b3b-42c0-4922-b8e2-3a2a371b1966 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f06317949284dcc935e92131e094525', 'request_id': 'req-d5bd2b3b-42c0-4922-b8e2-3a2a371b1966'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:31.352283 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-d5bd2b3b-42c0-4922-b8e2-3a2a371b1966 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f06317949284dcc935e92131e094525', 'request_id': 'req-d5bd2b3b-42c0-4922-b8e2-3a2a371b1966'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:31.362698 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-d5bd2b3b-42c0-4922-b8e2-3a2a371b1966 admin admin] The token cache is being invalidate because project 4f06317949284dcc935e92131e094525 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:55:31.362985 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-d5bd2b3b-42c0-4922-b8e2-3a2a371b1966 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:31.363676 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-d5bd2b3b-42c0-4922-b8e2-3a2a371b1966 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:31.365481 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1065/2149] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:55:31 2022] DELETE /identity/v3/projects/4f06317949284dcc935e92131e094525 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:31.370320 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704824fc-9d78-4962-b973-f04cbc9a0bb8 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:31.370832 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704824fc-9d78-4962-b973-f04cbc9a0bb8 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:31.370937 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704824fc-9d78-4962-b973-f04cbc9a0bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:31.390346 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-704824fc-9d78-4962-b973-f04cbc9a0bb8 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:31.391594 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-704824fc-9d78-4962-b973-f04cbc9a0bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:31.444372 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:31.453787 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1085/2150] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:31.462237 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272ddbf-c947-44b4-8410-797c3198aa5f None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:31.462528 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272ddbf-c947-44b4-8410-797c3198aa5f None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:31.462899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272ddbf-c947-44b4-8410-797c3198aa5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:31.483612 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-2272ddbf-c947-44b4-8410-797c3198aa5f None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:31.484984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-2272ddbf-c947-44b4-8410-797c3198aa5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:31.493737 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1066/2151] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:31.498218 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704824fc-9d78-4962-b973-f04cbc9a0bb8 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:31.508575 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704824fc-9d78-4962-b973-f04cbc9a0bb8 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:31.513130 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d7df4-eabc-4d12-a9e8-4bd674773737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:31.513727 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d7df4-eabc-4d12-a9e8-4bd674773737 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:31.514007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d7df4-eabc-4d12-a9e8-4bd674773737 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:31.514278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d7df4-eabc-4d12-a9e8-4bd674773737 admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:31.514919 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d7df4-eabc-4d12-a9e8-4bd674773737 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:55:31.515221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d7df4-eabc-4d12-a9e8-4bd674773737 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:31.516861 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d7df4-eabc-4d12-a9e8-4bd674773737 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:31.528635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1086/2152] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:55:31.533171 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272ddbf-c947-44b4-8410-797c3198aa5f None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:31.543344 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272ddbf-c947-44b4-8410-797c3198aa5f None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:31.548446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e18c39-ee6b-433b-b4e9-1738d82f7634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:31.549137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e18c39-ee6b-433b-b4e9-1738d82f7634 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:31.549421 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e18c39-ee6b-433b-b4e9-1738d82f7634 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:31.549807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e18c39-ee6b-433b-b4e9-1738d82f7634 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:31.550486 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e18c39-ee6b-433b-b4e9-1738d82f7634 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:31.552555 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e18c39-ee6b-433b-b4e9-1738d82f7634 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:31.692252 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1067/2153] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:55:31 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:55:31.693388 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d7df4-eabc-4d12-a9e8-4bd674773737 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:31.703539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d7df4-eabc-4d12-a9e8-4bd674773737 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:31.760093 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7310f527-1f59-4567-8473-6e17a8f89803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:31.760586 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310f527-1f59-4567-8473-6e17a8f89803 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:31.760844 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310f527-1f59-4567-8473-6e17a8f89803 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:31.761141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310f527-1f59-4567-8473-6e17a8f89803 admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:31.762008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7310f527-1f59-4567-8473-6e17a8f89803 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:31.763772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7310f527-1f59-4567-8473-6e17a8f89803 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:31.837181 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1087/2154] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:55:31 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:55:31.840614 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e18c39-ee6b-433b-b4e9-1738d82f7634 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:31.851134 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e18c39-ee6b-433b-b4e9-1738d82f7634 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:31.856097 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67279918-9b3b-4cd6-8c79-0905dec77df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:31.856740 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67279918-9b3b-4cd6-8c79-0905dec77df3 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:31.857023 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67279918-9b3b-4cd6-8c79-0905dec77df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:31.857300 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67279918-9b3b-4cd6-8c79-0905dec77df3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:31.857947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67279918-9b3b-4cd6-8c79-0905dec77df3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:31.859857 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67279918-9b3b-4cd6-8c79-0905dec77df3 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:31.872698 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1068/2155] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:55:31.874468 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7310f527-1f59-4567-8473-6e17a8f89803 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:31.885256 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7310f527-1f59-4567-8473-6e17a8f89803 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:31.890599 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608e3ae-8703-4a23-8226-424d9110f493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:31.891294 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608e3ae-8703-4a23-8226-424d9110f493 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:31.891578 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608e3ae-8703-4a23-8226-424d9110f493 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:31.891856 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608e3ae-8703-4a23-8226-424d9110f493 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:31.892550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a608e3ae-8703-4a23-8226-424d9110f493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:31.894234 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a608e3ae-8703-4a23-8226-424d9110f493 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:31.905467 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1088/2156] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 00:55:31.909706 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67279918-9b3b-4cd6-8c79-0905dec77df3 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:31.916450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:31.921911 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67279918-9b3b-4cd6-8c79-0905dec77df3 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:31.927349 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae726a1-8d15-4092-aa0d-986c58ae5607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:31.928002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae726a1-8d15-4092-aa0d-986c58ae5607 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:31.928281 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae726a1-8d15-4092-aa0d-986c58ae5607 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:31.928641 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae726a1-8d15-4092-aa0d-986c58ae5607 admin admin] PATH_INFO: `/v3/projects/c5ed426010bd4fc4973e1f5bec79883f/users/640b9b323b5f4c65a63e6b79a75aeb41/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:31.944963 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae726a1-8d15-4092-aa0d-986c58ae5607 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5ed426010bd4fc4973e1f5bec79883f, user_id=640b9b323b5f4c65a63e6b79a75aeb41, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:31.947645 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae726a1-8d15-4092-aa0d-986c58ae5607 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:32.041661 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1069/2157] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:55:31 2022] PUT /identity/v3/projects/c5ed426010bd4fc4973e1f5bec79883f/users/640b9b323b5f4c65a63e6b79a75aeb41/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:55:32.053848 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f70c7-e5ca-437e-aa82-7d3898af1cd5 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:32.054130 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f70c7-e5ca-437e-aa82-7d3898af1cd5 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:32.054401 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f70c7-e5ca-437e-aa82-7d3898af1cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:32.082905 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-bf6f70c7-e5ca-437e-aa82-7d3898af1cd5 None None] MFA Rules not processed for user `640b9b323b5f4c65a63e6b79a75aeb41`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:32.084119 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-bf6f70c7-e5ca-437e-aa82-7d3898af1cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:32.155145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1089/2158] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:32.169711 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae726a1-8d15-4092-aa0d-986c58ae5607 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:32.171486 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-aae726a1-8d15-4092-aa0d-986c58ae5607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:32.233779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae726a1-8d15-4092-aa0d-986c58ae5607 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:32.237594 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178ca60-155d-4d7b-80f8-cbd472b22dd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:32.238199 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178ca60-155d-4d7b-80f8-cbd472b22dd4 service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:32.238483 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178ca60-155d-4d7b-80f8-cbd472b22dd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:32.238757 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178ca60-155d-4d7b-80f8-cbd472b22dd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:32.247103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2178ca60-155d-4d7b-80f8-cbd472b22dd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:32.248564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2178ca60-155d-4d7b-80f8-cbd472b22dd4 service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:32.258060 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1070/2159] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:55:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:55:32.279267 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f70c7-e5ca-437e-aa82-7d3898af1cd5 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:32.344494 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:32.349926 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f70c7-e5ca-437e-aa82-7d3898af1cd5 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:32.356017 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9977ba-0895-4334-8126-a126543dd381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:32.356017 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9977ba-0895-4334-8126-a126543dd381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:32.356017 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9977ba-0895-4334-8126-a126543dd381 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:32.356017 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9977ba-0895-4334-8126-a126543dd381 admin admin] PATH_INFO: `/v3/users/640b9b323b5f4c65a63e6b79a75aeb41` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:32.357209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9977ba-0895-4334-8126-a126543dd381 admin admin] RBAC: Authorizing `identity:delete_user(user_id=640b9b323b5f4c65a63e6b79a75aeb41)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:32.359148 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9977ba-0895-4334-8126-a126543dd381 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:32.380829 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178ca60-155d-4d7b-80f8-cbd472b22dd4 service cinder] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:32.382215 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-2178ca60-155d-4d7b-80f8-cbd472b22dd4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:32.396186 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178ca60-155d-4d7b-80f8-cbd472b22dd4 service cinder] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:32.400742 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc5c3aa-a228-4eee-a070-b07ba3d35f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:32.401363 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5c3aa-a228-4eee-a070-b07ba3d35f6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:32.401640 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5c3aa-a228-4eee-a070-b07ba3d35f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:32.401909 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5c3aa-a228-4eee-a070-b07ba3d35f6a admin admin] PATH_INFO: `/v3/users/cfd8cabf83b54550a8f816a108a56fa8` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:32.403039 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc5c3aa-a228-4eee-a070-b07ba3d35f6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfd8cabf83b54550a8f816a108a56fa8)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:32.404985 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc5c3aa-a228-4eee-a070-b07ba3d35f6a admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:32.540500 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-be9977ba-0895-4334-8126-a126543dd381 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '640b9b323b5f4c65a63e6b79a75aeb41'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:32.541086 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-be9977ba-0895-4334-8126-a126543dd381 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '640b9b323b5f4c65a63e6b79a75aeb41'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:32.555342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-be9977ba-0895-4334-8126-a126543dd381 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '640b9b323b5f4c65a63e6b79a75aeb41'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:32.558488 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1cc5c3aa-a228-4eee-a070-b07ba3d35f6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfd8cabf83b54550a8f816a108a56fa8'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:32.571305 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1cc5c3aa-a228-4eee-a070-b07ba3d35f6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfd8cabf83b54550a8f816a108a56fa8'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:32.635012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-be9977ba-0895-4334-8126-a126543dd381 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '640b9b323b5f4c65a63e6b79a75aeb41'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:32.635641 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-be9977ba-0895-4334-8126-a126543dd381 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '640b9b323b5f4c65a63e6b79a75aeb41'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:32.648428 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1090/2160] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:55:32 2022] DELETE /identity/v3/users/640b9b323b5f4c65a63e6b79a75aeb41 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:32.652733 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e72dca-7688-4716-af90-accce183a2d8 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:32.653030 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e72dca-7688-4716-af90-accce183a2d8 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:32.653312 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e72dca-7688-4716-af90-accce183a2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:32.671839 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-57e72dca-7688-4716-af90-accce183a2d8 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:32.673100 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-57e72dca-7688-4716-af90-accce183a2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:32.726878 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1cc5c3aa-a228-4eee-a070-b07ba3d35f6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfd8cabf83b54550a8f816a108a56fa8'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:32.727550 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1cc5c3aa-a228-4eee-a070-b07ba3d35f6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfd8cabf83b54550a8f816a108a56fa8'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:32.728175 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1cc5c3aa-a228-4eee-a070-b07ba3d35f6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfd8cabf83b54550a8f816a108a56fa8'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:32.732146 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1091/2161] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:55:32.742227 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e72dca-7688-4716-af90-accce183a2d8 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:32.743580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-57e72dca-7688-4716-af90-accce183a2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:32.745229 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1071/2162] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:55:32 2022] DELETE /identity/v3/users/cfd8cabf83b54550a8f816a108a56fa8 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:55:32.749527 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4004ae3-2a5f-4a71-a9f5-37855e9094ea None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:32.749810 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4004ae3-2a5f-4a71-a9f5-37855e9094ea None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:32.750088 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4004ae3-2a5f-4a71-a9f5-37855e9094ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:32.769728 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-f4004ae3-2a5f-4a71-a9f5-37855e9094ea None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:32.770957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-f4004ae3-2a5f-4a71-a9f5-37855e9094ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:32.832659 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:32.844791 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1072/2163] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:32.857023 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4004ae3-2a5f-4a71-a9f5-37855e9094ea None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:32.858450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-f4004ae3-2a5f-4a71-a9f5-37855e9094ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:32.921167 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4004ae3-2a5f-4a71-a9f5-37855e9094ea None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:32.924947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8684014-3449-4f5b-9d04-2864c6b58274 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:32.925589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8684014-3449-4f5b-9d04-2864c6b58274 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:32.925833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e72dca-7688-4716-af90-accce183a2d8 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:32.925939 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8684014-3449-4f5b-9d04-2864c6b58274 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:32.926163 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8684014-3449-4f5b-9d04-2864c6b58274 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:32.929431 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec05b2-10cb-4ec2-b3eb-2f4d40b01235 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:32.930018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec05b2-10cb-4ec2-b3eb-2f4d40b01235 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:32.930297 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec05b2-10cb-4ec2-b3eb-2f4d40b01235 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:32.930575 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec05b2-10cb-4ec2-b3eb-2f4d40b01235 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:32.935107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8684014-3449-4f5b-9d04-2864c6b58274 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:32.936572 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8684014-3449-4f5b-9d04-2864c6b58274 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:32.939560 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ec05b2-10cb-4ec2-b3eb-2f4d40b01235 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:32.941059 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ec05b2-10cb-4ec2-b3eb-2f4d40b01235 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:32.946974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1073/2164] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:55:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:55:32.950534 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1092/2165] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:55:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:55:33.623339 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8684014-3449-4f5b-9d04-2864c6b58274 service neutron] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:33.626051 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-a8684014-3449-4f5b-9d04-2864c6b58274 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:33.638240 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8684014-3449-4f5b-9d04-2864c6b58274 service neutron] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:33.651192 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd062c-4bc5-4d0f-9cb2-45ae82fc20e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:33.651192 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd062c-4bc5-4d0f-9cb2-45ae82fc20e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:33.651192 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd062c-4bc5-4d0f-9cb2-45ae82fc20e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:33.651192 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd062c-4bc5-4d0f-9cb2-45ae82fc20e6 admin admin] PATH_INFO: `/v3/projects/31471c7bbea34dc0b250e444c70f9baa` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:33.651947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd062c-4bc5-4d0f-9cb2-45ae82fc20e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31471c7bbea34dc0b250e444c70f9baa)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:33.654917 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec05b2-10cb-4ec2-b3eb-2f4d40b01235 service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:33.655377 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd062c-4bc5-4d0f-9cb2-45ae82fc20e6 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:33.656424 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-26ec05b2-10cb-4ec2-b3eb-2f4d40b01235 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:33.672462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec05b2-10cb-4ec2-b3eb-2f4d40b01235 service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:33.686089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2bd151-3c32-4f04-8ee0-815b3e09092a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:33.686089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bd151-3c32-4f04-8ee0-815b3e09092a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:33.686089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bd151-3c32-4f04-8ee0-815b3e09092a admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:33.686089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bd151-3c32-4f04-8ee0-815b3e09092a admin admin] PATH_INFO: `/v3/projects/c5ed426010bd4fc4973e1f5bec79883f` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:33.686850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2bd151-3c32-4f04-8ee0-815b3e09092a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5ed426010bd4fc4973e1f5bec79883f)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:33.686850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2bd151-3c32-4f04-8ee0-815b3e09092a admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:33.699181 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:33.790824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-73fd062c-4bc5-4d0f-9cb2-45ae82fc20e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31471c7bbea34dc0b250e444c70f9baa', 'request_id': 'req-73fd062c-4bc5-4d0f-9cb2-45ae82fc20e6'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:33.814188 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-8d2bd151-3c32-4f04-8ee0-815b3e09092a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5ed426010bd4fc4973e1f5bec79883f', 'request_id': 'req-8d2bd151-3c32-4f04-8ee0-815b3e09092a'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:33.881213 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-73fd062c-4bc5-4d0f-9cb2-45ae82fc20e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31471c7bbea34dc0b250e444c70f9baa', 'request_id': 'req-73fd062c-4bc5-4d0f-9cb2-45ae82fc20e6'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:33.888488 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-73fd062c-4bc5-4d0f-9cb2-45ae82fc20e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31471c7bbea34dc0b250e444c70f9baa', 'request_id': 'req-73fd062c-4bc5-4d0f-9cb2-45ae82fc20e6'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:33.899780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-73fd062c-4bc5-4d0f-9cb2-45ae82fc20e6 admin admin] The token cache is being invalidate because project 31471c7bbea34dc0b250e444c70f9baa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:55:33.900067 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-73fd062c-4bc5-4d0f-9cb2-45ae82fc20e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:33.900827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-73fd062c-4bc5-4d0f-9cb2-45ae82fc20e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:33.902380 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1074/2166] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:55:33 2022] DELETE /identity/v3/projects/31471c7bbea34dc0b250e444c70f9baa => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:33.908055 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd062c-4bc5-4d0f-9cb2-45ae82fc20e6 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:33.908055 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-73fd062c-4bc5-4d0f-9cb2-45ae82fc20e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:33.972634 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd062c-4bc5-4d0f-9cb2-45ae82fc20e6 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:33.972877 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-8d2bd151-3c32-4f04-8ee0-815b3e09092a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5ed426010bd4fc4973e1f5bec79883f', 'request_id': 'req-8d2bd151-3c32-4f04-8ee0-815b3e09092a'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:33.979698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-8d2bd151-3c32-4f04-8ee0-815b3e09092a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5ed426010bd4fc4973e1f5bec79883f', 'request_id': 'req-8d2bd151-3c32-4f04-8ee0-815b3e09092a'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:33.993449 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-8d2bd151-3c32-4f04-8ee0-815b3e09092a admin admin] The token cache is being invalidate because project c5ed426010bd4fc4973e1f5bec79883f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:55:33.993743 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-8d2bd151-3c32-4f04-8ee0-815b3e09092a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:33.994417 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-8d2bd151-3c32-4f04-8ee0-815b3e09092a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:33.996048 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1093/2167] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:55:33 2022] DELETE /identity/v3/projects/c5ed426010bd4fc4973e1f5bec79883f => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:55:33.996166 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:34.001706 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6084e61-e5ef-4906-b0f2-7f190178c5fb None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:34.002250 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6084e61-e5ef-4906-b0f2-7f190178c5fb None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:34.003506 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6084e61-e5ef-4906-b0f2-7f190178c5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:34.025518 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-d6084e61-e5ef-4906-b0f2-7f190178c5fb None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:34.027616 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-d6084e61-e5ef-4906-b0f2-7f190178c5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:34.040834 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578b5a8-6395-483b-bdb0-2f2d9beebe74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:34.041972 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578b5a8-6395-483b-bdb0-2f2d9beebe74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:34.042462 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578b5a8-6395-483b-bdb0-2f2d9beebe74 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:34.042912 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578b5a8-6395-483b-bdb0-2f2d9beebe74 admin admin] PATH_INFO: `/v3/users/42aeb986e27e4f1fa956ceabd15bea53` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:34.044295 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578b5a8-6395-483b-bdb0-2f2d9beebe74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42aeb986e27e4f1fa956ceabd15bea53)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:34.046888 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578b5a8-6395-483b-bdb0-2f2d9beebe74 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:34.095559 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1094/2168] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:55:34.104968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c037a2-b30f-4dc1-9eca-b1678fbfd00c None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:34.105257 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c037a2-b30f-4dc1-9eca-b1678fbfd00c None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:34.105537 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c037a2-b30f-4dc1-9eca-b1678fbfd00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:34.127169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-f3c037a2-b30f-4dc1-9eca-b1678fbfd00c None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:34.129017 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f3c037a2-b30f-4dc1-9eca-b1678fbfd00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:34.137818 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1095/2169] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:55:34.143173 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c037a2-b30f-4dc1-9eca-b1678fbfd00c None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:34.153580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c037a2-b30f-4dc1-9eca-b1678fbfd00c None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:34.158147 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4fa384-2afe-4208-b065-ffa817a9e3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:34.158764 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4fa384-2afe-4208-b065-ffa817a9e3a8 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:34.159051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4fa384-2afe-4208-b065-ffa817a9e3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:34.159320 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4fa384-2afe-4208-b065-ffa817a9e3a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:34.159976 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4fa384-2afe-4208-b065-ffa817a9e3a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:55:34.160300 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4fa384-2afe-4208-b065-ffa817a9e3a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:34.162198 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4fa384-2afe-4208-b065-ffa817a9e3a8 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:34.177256 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1096/2170] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:55:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:55:34.181577 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4fa384-2afe-4208-b065-ffa817a9e3a8 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:34.194817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4fa384-2afe-4208-b065-ffa817a9e3a8 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:34.199566 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5918a4-619f-4da4-8b00-308e45b12476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:34.200239 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5918a4-619f-4da4-8b00-308e45b12476 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:34.200614 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5918a4-619f-4da4-8b00-308e45b12476 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:34.200885 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5918a4-619f-4da4-8b00-308e45b12476 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:34.201693 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5918a4-619f-4da4-8b00-308e45b12476 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:34.203585 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5918a4-619f-4da4-8b00-308e45b12476 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:34.277618 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f578b5a8-6395-483b-bdb0-2f2d9beebe74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42aeb986e27e4f1fa956ceabd15bea53'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:34.309192 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f578b5a8-6395-483b-bdb0-2f2d9beebe74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42aeb986e27e4f1fa956ceabd15bea53'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:34.312333 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1097/2171] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:55:34 2022] POST /identity/v3/projects => generated 371 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:55:34.317812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5918a4-619f-4da4-8b00-308e45b12476 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:34.392050 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5918a4-619f-4da4-8b00-308e45b12476 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:34.397078 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a4843-9f46-4d14-85be-c669bbf24c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:34.397685 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a4843-9f46-4d14-85be-c669bbf24c8b admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:34.397966 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a4843-9f46-4d14-85be-c669bbf24c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:34.398234 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a4843-9f46-4d14-85be-c669bbf24c8b admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:34.398936 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a4843-9f46-4d14-85be-c669bbf24c8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:34.400942 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a4843-9f46-4d14-85be-c669bbf24c8b admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:34.408782 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:34.424087 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f578b5a8-6395-483b-bdb0-2f2d9beebe74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42aeb986e27e4f1fa956ceabd15bea53'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:34.425442 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f578b5a8-6395-483b-bdb0-2f2d9beebe74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42aeb986e27e4f1fa956ceabd15bea53'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:34.426104 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f578b5a8-6395-483b-bdb0-2f2d9beebe74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42aeb986e27e4f1fa956ceabd15bea53'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:34.443657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1075/2172] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:55:33 2022] DELETE /identity/v3/users/42aeb986e27e4f1fa956ceabd15bea53 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:55:34.520705 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1098/2173] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:55:34 2022] POST /identity/v3/users => generated 286 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:55:34.521959 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578b5a8-6395-483b-bdb0-2f2d9beebe74 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:34.523462 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-f578b5a8-6395-483b-bdb0-2f2d9beebe74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:34.588239 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578b5a8-6395-483b-bdb0-2f2d9beebe74 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:34.593046 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9556c4-1bf6-40e4-a5d3-247a6c875629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:34.593586 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9556c4-1bf6-40e4-a5d3-247a6c875629 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:34.593865 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9556c4-1bf6-40e4-a5d3-247a6c875629 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:34.594136 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9556c4-1bf6-40e4-a5d3-247a6c875629 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:34.594774 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9556c4-1bf6-40e4-a5d3-247a6c875629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:34.597006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9556c4-1bf6-40e4-a5d3-247a6c875629 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:34.606594 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1076/2174] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:55:34.610810 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a4843-9f46-4d14-85be-c669bbf24c8b admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:34.622277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a4843-9f46-4d14-85be-c669bbf24c8b admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:34.626986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51158c7b-2313-4fa9-910e-871336ef60d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:34.628352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51158c7b-2313-4fa9-910e-871336ef60d0 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:34.628689 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51158c7b-2313-4fa9-910e-871336ef60d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:34.628948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51158c7b-2313-4fa9-910e-871336ef60d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:34.629603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51158c7b-2313-4fa9-910e-871336ef60d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:34.631200 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51158c7b-2313-4fa9-910e-871336ef60d0 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:34.642072 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1099/2175] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 00:55:34.646364 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9556c4-1bf6-40e4-a5d3-247a6c875629 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:34.656971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9556c4-1bf6-40e4-a5d3-247a6c875629 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:34.662909 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ccbfd-2b94-4cc5-846a-3a28641163fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:34.663656 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ccbfd-2b94-4cc5-846a-3a28641163fb admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:34.663996 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ccbfd-2b94-4cc5-846a-3a28641163fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:34.664335 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ccbfd-2b94-4cc5-846a-3a28641163fb admin admin] PATH_INFO: `/v3/projects/29601c525a8d42878760f5ae2c4eb580/users/d959f97aff1e4cf0b6b76d7b4ad9705f/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:34.685299 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ccbfd-2b94-4cc5-846a-3a28641163fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=29601c525a8d42878760f5ae2c4eb580, user_id=d959f97aff1e4cf0b6b76d7b4ad9705f, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:34.687984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ccbfd-2b94-4cc5-846a-3a28641163fb admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:34.773066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1077/2176] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:55:34 2022] PUT /identity/v3/projects/29601c525a8d42878760f5ae2c4eb580/users/d959f97aff1e4cf0b6b76d7b4ad9705f/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:34.783438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b76c59-1df4-4eee-b8f5-d27f4ff0f7b7 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:34.784546 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b76c59-1df4-4eee-b8f5-d27f4ff0f7b7 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:34.784546 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b76c59-1df4-4eee-b8f5-d27f4ff0f7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:34.814700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-a5b76c59-1df4-4eee-b8f5-d27f4ff0f7b7 None None] MFA Rules not processed for user `d959f97aff1e4cf0b6b76d7b4ad9705f`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:34.815960 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a5b76c59-1df4-4eee-b8f5-d27f4ff0f7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:34.850122 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ccbfd-2b94-4cc5-846a-3a28641163fb admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:34.851950 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-688ccbfd-2b94-4cc5-846a-3a28641163fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:34.876553 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:34.906577 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1100/2177] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:34.918905 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b76c59-1df4-4eee-b8f5-d27f4ff0f7b7 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:34.921465 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a5b76c59-1df4-4eee-b8f5-d27f4ff0f7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:34.926233 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ccbfd-2b94-4cc5-846a-3a28641163fb admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:34.930997 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e61bb-8e6d-4c95-9301-1550fda7a604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:34.931630 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e61bb-8e6d-4c95-9301-1550fda7a604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:34.931909 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e61bb-8e6d-4c95-9301-1550fda7a604 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:34.932386 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e61bb-8e6d-4c95-9301-1550fda7a604 admin admin] PATH_INFO: `/v3/projects/9bd31142621142e4b57095f5ad1c7019` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:34.933525 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e61bb-8e6d-4c95-9301-1550fda7a604 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bd31142621142e4b57095f5ad1c7019)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:34.935470 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e61bb-8e6d-4c95-9301-1550fda7a604 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:35.001443 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b76c59-1df4-4eee-b8f5-d27f4ff0f7b7 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:35.005240 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47917a99-3ca4-4e8e-a20a-41d745248c2b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abed7ebb0aae4e4693ce08e51a149312', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:35.005850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47917a99-3ca4-4e8e-a20a-41d745248c2b service glance] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:35.006159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47917a99-3ca4-4e8e-a20a-41d745248c2b service glance] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:35.006460 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47917a99-3ca4-4e8e-a20a-41d745248c2b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:35.014850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47917a99-3ca4-4e8e-a20a-41d745248c2b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:35.016276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47917a99-3ca4-4e8e-a20a-41d745248c2b service glance] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:35.026178 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1101/2178] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:55:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:55:35.042642 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-d45e61bb-8e6d-4c95-9301-1550fda7a604 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bd31142621142e4b57095f5ad1c7019', 'request_id': 'req-d45e61bb-8e6d-4c95-9301-1550fda7a604'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:35.120423 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-d45e61bb-8e6d-4c95-9301-1550fda7a604 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bd31142621142e4b57095f5ad1c7019', 'request_id': 'req-d45e61bb-8e6d-4c95-9301-1550fda7a604'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:35.127750 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-d45e61bb-8e6d-4c95-9301-1550fda7a604 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bd31142621142e4b57095f5ad1c7019', 'request_id': 'req-d45e61bb-8e6d-4c95-9301-1550fda7a604'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:35.140816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-d45e61bb-8e6d-4c95-9301-1550fda7a604 admin admin] The token cache is being invalidate because project 9bd31142621142e4b57095f5ad1c7019 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:55:35.141109 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-d45e61bb-8e6d-4c95-9301-1550fda7a604 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:35.141781 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-d45e61bb-8e6d-4c95-9301-1550fda7a604 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:35.144148 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1078/2179] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:55:34 2022] DELETE /identity/v3/projects/9bd31142621142e4b57095f5ad1c7019 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:35.147936 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47917a99-3ca4-4e8e-a20a-41d745248c2b service glance] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:35.160023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47917a99-3ca4-4e8e-a20a-41d745248c2b service glance] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:35.209070 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:35.219452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28cf79-b85e-4122-a4e0-3585e9f47ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:35.220091 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28cf79-b85e-4122-a4e0-3585e9f47ca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:35.220369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28cf79-b85e-4122-a4e0-3585e9f47ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:35.220679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28cf79-b85e-4122-a4e0-3585e9f47ca4 admin admin] PATH_INFO: `/v3/users/a7aca84d1d3048c5aa0fd7546e09ef5b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:35.222536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce28cf79-b85e-4122-a4e0-3585e9f47ca4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7aca84d1d3048c5aa0fd7546e09ef5b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:35.224756 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce28cf79-b85e-4122-a4e0-3585e9f47ca4 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:35.385811 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ce28cf79-b85e-4122-a4e0-3585e9f47ca4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7aca84d1d3048c5aa0fd7546e09ef5b'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:35.387082 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ce28cf79-b85e-4122-a4e0-3585e9f47ca4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7aca84d1d3048c5aa0fd7546e09ef5b'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:35.420867 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ce28cf79-b85e-4122-a4e0-3585e9f47ca4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7aca84d1d3048c5aa0fd7546e09ef5b'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:35.505539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ce28cf79-b85e-4122-a4e0-3585e9f47ca4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7aca84d1d3048c5aa0fd7546e09ef5b'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:35.506983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ce28cf79-b85e-4122-a4e0-3585e9f47ca4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7aca84d1d3048c5aa0fd7546e09ef5b'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:35.520122 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1102/2180] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:55:35 2022] DELETE /identity/v3/users/a7aca84d1d3048c5aa0fd7546e09ef5b => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:55:35.933655 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e61bb-8e6d-4c95-9301-1550fda7a604 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:35.935291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-d45e61bb-8e6d-4c95-9301-1550fda7a604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:35.962561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28cf79-b85e-4122-a4e0-3585e9f47ca4 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:35.965339 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-ce28cf79-b85e-4122-a4e0-3585e9f47ca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:36.020858 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e61bb-8e6d-4c95-9301-1550fda7a604 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:36.084965 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4ae86-c153-4823-87d0-d69f83542c90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:36.085591 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4ae86-c153-4823-87d0-d69f83542c90 service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:36.085867 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4ae86-c153-4823-87d0-d69f83542c90 service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:36.086136 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4ae86-c153-4823-87d0-d69f83542c90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:36.088026 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-10c4ae86-c153-4823-87d0-d69f83542c90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:36.094205 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28cf79-b85e-4122-a4e0-3585e9f47ca4 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:36.104268 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eba1a0-b88c-45f7-8bce-3c652a046c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:36.105106 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eba1a0-b88c-45f7-8bce-3c652a046c64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:36.105512 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eba1a0-b88c-45f7-8bce-3c652a046c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:36.105894 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eba1a0-b88c-45f7-8bce-3c652a046c64 admin admin] PATH_INFO: `/v3/projects/df3021fb2a8d4111880c8f3d80d10333` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:36.107178 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eba1a0-b88c-45f7-8bce-3c652a046c64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df3021fb2a8d4111880c8f3d80d10333)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:36.109274 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eba1a0-b88c-45f7-8bce-3c652a046c64 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:36.110389 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:36.168592 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c4ae86-c153-4823-87d0-d69f83542c90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:36.170650 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c4ae86-c153-4823-87d0-d69f83542c90 service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:36.183400 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1079/2181] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:55:35 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:55:36.242200 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-d1eba1a0-b88c-45f7-8bce-3c652a046c64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df3021fb2a8d4111880c8f3d80d10333', 'request_id': 'req-d1eba1a0-b88c-45f7-8bce-3c652a046c64'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:36.333660 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-d1eba1a0-b88c-45f7-8bce-3c652a046c64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df3021fb2a8d4111880c8f3d80d10333', 'request_id': 'req-d1eba1a0-b88c-45f7-8bce-3c652a046c64'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:36.340959 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-d1eba1a0-b88c-45f7-8bce-3c652a046c64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df3021fb2a8d4111880c8f3d80d10333', 'request_id': 'req-d1eba1a0-b88c-45f7-8bce-3c652a046c64'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:36.353445 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:36.367271 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-d1eba1a0-b88c-45f7-8bce-3c652a046c64 admin admin] The token cache is being invalidate because project df3021fb2a8d4111880c8f3d80d10333 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:55:36.367621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-d1eba1a0-b88c-45f7-8bce-3c652a046c64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:36.368242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-d1eba1a0-b88c-45f7-8bce-3c652a046c64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:55:36.370359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1103/2182] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:55:35 2022] DELETE /identity/v3/projects/df3021fb2a8d4111880c8f3d80d10333 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:36.375272 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e62fa8-366d-4104-acc0-72cad60f398a None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:36.375735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e62fa8-366d-4104-acc0-72cad60f398a None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:36.376016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e62fa8-366d-4104-acc0-72cad60f398a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:36.398731 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-d6e62fa8-366d-4104-acc0-72cad60f398a None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:36.399944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-d6e62fa8-366d-4104-acc0-72cad60f398a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:36.490660 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1080/2183] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:36.498215 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e76d7-be76-4e51-933e-9f9155d9f14f None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:36.498700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e76d7-be76-4e51-933e-9f9155d9f14f None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:36.499024 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e76d7-be76-4e51-933e-9f9155d9f14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:36.521583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-1e9e76d7-be76-4e51-933e-9f9155d9f14f None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:36.523547 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-1e9e76d7-be76-4e51-933e-9f9155d9f14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:36.536307 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1104/2184] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:36.538077 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e62fa8-366d-4104-acc0-72cad60f398a None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:36.550283 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e62fa8-366d-4104-acc0-72cad60f398a None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:36.555525 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e4d69-0d61-4fd2-a8d8-a7002162b1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:36.558083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e4d69-0d61-4fd2-a8d8-a7002162b1c6 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:36.558083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e4d69-0d61-4fd2-a8d8-a7002162b1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:36.558083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e4d69-0d61-4fd2-a8d8-a7002162b1c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:36.558083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e4d69-0d61-4fd2-a8d8-a7002162b1c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:55:36.558083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e4d69-0d61-4fd2-a8d8-a7002162b1c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:36.559949 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e4d69-0d61-4fd2-a8d8-a7002162b1c6 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:36.576916 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1081/2185] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:55:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:55:36.578334 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e76d7-be76-4e51-933e-9f9155d9f14f None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:36.590138 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e76d7-be76-4e51-933e-9f9155d9f14f None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:36.595064 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea9f1db-06e8-4537-b5e3-ab49cd901b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:36.595705 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9f1db-06e8-4537-b5e3-ab49cd901b7f admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:36.596324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9f1db-06e8-4537-b5e3-ab49cd901b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:36.597154 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9f1db-06e8-4537-b5e3-ab49cd901b7f admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:36.597819 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea9f1db-06e8-4537-b5e3-ab49cd901b7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:36.599893 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea9f1db-06e8-4537-b5e3-ab49cd901b7f admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:36.720763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1105/2186] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:55:36 2022] POST /identity/v3/projects => generated 379 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:55:36.725983 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e4d69-0d61-4fd2-a8d8-a7002162b1c6 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:36.738428 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e4d69-0d61-4fd2-a8d8-a7002162b1c6 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:36.813940 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b6245-dab4-4e5e-8602-c2867637e9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:36.814656 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b6245-dab4-4e5e-8602-c2867637e9ab admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:36.815026 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b6245-dab4-4e5e-8602-c2867637e9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:36.815361 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b6245-dab4-4e5e-8602-c2867637e9ab admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:36.816071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b6245-dab4-4e5e-8602-c2867637e9ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:36.818289 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b6245-dab4-4e5e-8602-c2867637e9ab admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:36.956465 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea9f1db-06e8-4537-b5e3-ab49cd901b7f admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:36.959302 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1082/2187] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:55:36 2022] POST /identity/v3/users => generated 290 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:55:36.974870 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea9f1db-06e8-4537-b5e3-ab49cd901b7f admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:36.981785 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6387f8-2897-47b4-a682-f2e734c87bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:36.982412 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6387f8-2897-47b4-a682-f2e734c87bbf admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:36.982688 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6387f8-2897-47b4-a682-f2e734c87bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:36.982951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6387f8-2897-47b4-a682-f2e734c87bbf admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:36.983661 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6387f8-2897-47b4-a682-f2e734c87bbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:36.985232 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6387f8-2897-47b4-a682-f2e734c87bbf admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:37.001827 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1106/2188] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:55:37.002646 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b6245-dab4-4e5e-8602-c2867637e9ab admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:37.013534 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b6245-dab4-4e5e-8602-c2867637e9ab admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:37.019349 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77979a-704f-4dfb-b9e3-a80d1eeb6e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:37.021469 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77979a-704f-4dfb-b9e3-a80d1eeb6e62 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:37.021469 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77979a-704f-4dfb-b9e3-a80d1eeb6e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:37.021469 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77979a-704f-4dfb-b9e3-a80d1eeb6e62 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:37.021469 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c77979a-704f-4dfb-b9e3-a80d1eeb6e62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:37.022894 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c77979a-704f-4dfb-b9e3-a80d1eeb6e62 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:37.039490 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1083/2189] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 00:55:37.040886 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6387f8-2897-47b4-a682-f2e734c87bbf admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:37.054581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6387f8-2897-47b4-a682-f2e734c87bbf admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:37.059349 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe33dcb-387c-4087-8887-03fdb3f53025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:37.060027 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe33dcb-387c-4087-8887-03fdb3f53025 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:37.060307 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe33dcb-387c-4087-8887-03fdb3f53025 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:37.060625 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe33dcb-387c-4087-8887-03fdb3f53025 admin admin] PATH_INFO: `/v3/projects/ca3e24d51bdf44c5ad7219fa57d3d23b/users/cd45f1813efc4926a30454aaa108ac47/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:37.077752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe33dcb-387c-4087-8887-03fdb3f53025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca3e24d51bdf44c5ad7219fa57d3d23b, user_id=cd45f1813efc4926a30454aaa108ac47, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:37.080750 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe33dcb-387c-4087-8887-03fdb3f53025 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:37.178928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1107/2190] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:55:37 2022] PUT /identity/v3/projects/ca3e24d51bdf44c5ad7219fa57d3d23b/users/cd45f1813efc4926a30454aaa108ac47/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:37.184071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8471e-54a4-462a-b795-b214224fcb75 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:37.184431 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8471e-54a4-462a-b795-b214224fcb75 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:37.184742 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8471e-54a4-462a-b795-b214224fcb75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:37.204215 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-b1d8471e-54a4-462a-b795-b214224fcb75 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:37.205408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-b1d8471e-54a4-462a-b795-b214224fcb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:37.218128 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:37.266745 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1084/2191] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:37.277054 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe33dcb-387c-4087-8887-03fdb3f53025 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:37.279793 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-efe33dcb-387c-4087-8887-03fdb3f53025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:37.338618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe33dcb-387c-4087-8887-03fdb3f53025 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:37.342488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50929ed-0e86-4d8c-a43c-9aae3fa14c55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:37.343140 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50929ed-0e86-4d8c-a43c-9aae3fa14c55 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:37.343457 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50929ed-0e86-4d8c-a43c-9aae3fa14c55 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:37.343768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50929ed-0e86-4d8c-a43c-9aae3fa14c55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:37.353773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50929ed-0e86-4d8c-a43c-9aae3fa14c55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:37.355211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50929ed-0e86-4d8c-a43c-9aae3fa14c55 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:37.364568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1108/2192] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:55:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:55:41.483350 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f597d-697b-4bc4-af55-cc6ebd957736 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:41.483350 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f597d-697b-4bc4-af55-cc6ebd957736 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:41.483350 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f597d-697b-4bc4-af55-cc6ebd957736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:41.503982 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-6f0f597d-697b-4bc4-af55-cc6ebd957736 None None] MFA Rules not processed for user `2f93774232c1479aa3e856bffb352140`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:41.505275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-6f0f597d-697b-4bc4-af55-cc6ebd957736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:41.604006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1085/2193] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:44.297729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f64d1f-368e-4070-935c-972e4bb60cb3 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:44.298342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f64d1f-368e-4070-935c-972e4bb60cb3 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:44.298342 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f64d1f-368e-4070-935c-972e4bb60cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:44.334767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-e8f64d1f-368e-4070-935c-972e4bb60cb3 None None] MFA Rules not processed for user `cd45f1813efc4926a30454aaa108ac47`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:44.336102 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-e8f64d1f-368e-4070-935c-972e4bb60cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:44.422716 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1109/2194] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:44.427267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f597d-697b-4bc4-af55-cc6ebd957736 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:44.446979 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f597d-697b-4bc4-af55-cc6ebd957736 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:44.454415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa71d34-f508-4d96-82ec-b98c0a1eda8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:44.455265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa71d34-f508-4d96-82ec-b98c0a1eda8c admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:44.455754 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa71d34-f508-4d96-82ec-b98c0a1eda8c admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:44.456736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa71d34-f508-4d96-82ec-b98c0a1eda8c admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:44.466281 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa71d34-f508-4d96-82ec-b98c0a1eda8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:44.466281 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa71d34-f508-4d96-82ec-b98c0a1eda8c admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:44.565849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1086/2195] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:55:44 2022] POST /identity/v3/projects => generated 379 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:55:44.576408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f64d1f-368e-4070-935c-972e4bb60cb3 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:44.589867 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f64d1f-368e-4070-935c-972e4bb60cb3 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:44.620913 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:44.654621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6123b5-e780-459a-81a4-a47dc53f51e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:44.654861 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6123b5-e780-459a-81a4-a47dc53f51e0 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:44.655166 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6123b5-e780-459a-81a4-a47dc53f51e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:44.655515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6123b5-e780-459a-81a4-a47dc53f51e0 admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:44.656269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6123b5-e780-459a-81a4-a47dc53f51e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:44.658755 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6123b5-e780-459a-81a4-a47dc53f51e0 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:44.749257 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1110/2196] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:55:44 2022] POST /identity/v3/users => generated 296 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:55:44.751338 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa71d34-f508-4d96-82ec-b98c0a1eda8c admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:44.765934 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa71d34-f508-4d96-82ec-b98c0a1eda8c admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:44.770564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31eefdc-e601-4247-a410-e570ff8f0710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:44.773357 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31eefdc-e601-4247-a410-e570ff8f0710 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:44.773357 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31eefdc-e601-4247-a410-e570ff8f0710 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:44.773357 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31eefdc-e601-4247-a410-e570ff8f0710 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:44.773357 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31eefdc-e601-4247-a410-e570ff8f0710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:44.774029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31eefdc-e601-4247-a410-e570ff8f0710 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:44.784269 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1087/2197] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:55:44.789807 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6123b5-e780-459a-81a4-a47dc53f51e0 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:44.801097 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6123b5-e780-459a-81a4-a47dc53f51e0 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:44.805830 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c4dd0-b822-45be-a01b-f8d57827ac8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:44.806522 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c4dd0-b822-45be-a01b-f8d57827ac8d admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:44.806806 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c4dd0-b822-45be-a01b-f8d57827ac8d admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:44.807084 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c4dd0-b822-45be-a01b-f8d57827ac8d admin admin] PATH_INFO: `/v3/projects/016ffb0e1f50431299ecff65e07bd180/users/b7e3d19238f64065b3191400f12534b6/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:44.829088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c4dd0-b822-45be-a01b-f8d57827ac8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=016ffb0e1f50431299ecff65e07bd180, user_id=b7e3d19238f64065b3191400f12534b6, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:44.829989 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c4dd0-b822-45be-a01b-f8d57827ac8d admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:44.938763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1111/2198] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:55:44 2022] PUT /identity/v3/projects/016ffb0e1f50431299ecff65e07bd180/users/b7e3d19238f64065b3191400f12534b6/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:44.940460 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31eefdc-e601-4247-a410-e570ff8f0710 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:44.951066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31eefdc-e601-4247-a410-e570ff8f0710 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:45.018133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c14be6-7285-4a6c-9b76-c3e0ca192d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:45.018489 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c14be6-7285-4a6c-9b76-c3e0ca192d66 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:45.018769 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c14be6-7285-4a6c-9b76-c3e0ca192d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:45.019042 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c14be6-7285-4a6c-9b76-c3e0ca192d66 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:45.019739 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c14be6-7285-4a6c-9b76-c3e0ca192d66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:45.021356 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c14be6-7285-4a6c-9b76-c3e0ca192d66 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:45.031651 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1088/2199] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 00:55:45.036322 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c4dd0-b822-45be-a01b-f8d57827ac8d admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:45.048749 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c4dd0-b822-45be-a01b-f8d57827ac8d admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:45.053680 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2d477-07ee-4066-9e64-003bfb6b4dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:45.054345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2d477-07ee-4066-9e64-003bfb6b4dea admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:45.054622 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2d477-07ee-4066-9e64-003bfb6b4dea admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:45.054892 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2d477-07ee-4066-9e64-003bfb6b4dea admin admin] PATH_INFO: `/v3/projects/016ffb0e1f50431299ecff65e07bd180/users/b7e3d19238f64065b3191400f12534b6/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:45.056941 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2d477-07ee-4066-9e64-003bfb6b4dea admin admin] RBAC: Authorizing `identity:create_grant(project_id=016ffb0e1f50431299ecff65e07bd180, user_id=b7e3d19238f64065b3191400f12534b6, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:45.065582 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2d477-07ee-4066-9e64-003bfb6b4dea admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:45.093502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1112/2200] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:55:45 2022] PUT /identity/v3/projects/016ffb0e1f50431299ecff65e07bd180/users/b7e3d19238f64065b3191400f12534b6/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:55:51.953782 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143d5be-fe96-4c7e-8119-14b374235e43 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:51.954324 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143d5be-fe96-4c7e-8119-14b374235e43 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:51.954324 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143d5be-fe96-4c7e-8119-14b374235e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:51.992763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-a143d5be-fe96-4c7e-8119-14b374235e43 None None] MFA Rules not processed for user `b7e3d19238f64065b3191400f12534b6`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:55:51.994257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-a143d5be-fe96-4c7e-8119-14b374235e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:52.012669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:55:52.104752 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1089/2201] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:55:51 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:55:52.123106 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2d477-07ee-4066-9e64-003bfb6b4dea admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:55:52.124947 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-8da2d477-07ee-4066-9e64-003bfb6b4dea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:55:52.226112 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2d477-07ee-4066-9e64-003bfb6b4dea admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:55:52.231994 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a5e0b-5445-40f0-ab52-1a3611bf324e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:55:52.232779 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a5e0b-5445-40f0-ab52-1a3611bf324e service nova] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:55:52.232971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a5e0b-5445-40f0-ab52-1a3611bf324e service nova] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:55:52.233225 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a5e0b-5445-40f0-ab52-1a3611bf324e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:55:52.304589 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a5e0b-5445-40f0-ab52-1a3611bf324e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:55:52.306752 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a5e0b-5445-40f0-ab52-1a3611bf324e service nova] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:55:52.319101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1113/2202] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:55:52 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:56:02.149276 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=21 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:02.314131 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=87,keystone:INSERT=5 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:05.259283 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a143d5be-fe96-4c7e-8119-14b374235e43 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:05.262815 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-a143d5be-fe96-4c7e-8119-14b374235e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:05.332073 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a143d5be-fe96-4c7e-8119-14b374235e43 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:05.337023 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd5a0c-71b3-4843-9e03-0ebe03cccdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:05.337541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd5a0c-71b3-4843-9e03-0ebe03cccdd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:05.337820 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd5a0c-71b3-4843-9e03-0ebe03cccdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:05.338094 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd5a0c-71b3-4843-9e03-0ebe03cccdd2 admin admin] PATH_INFO: `/v3/users/56765a3804124aa183444b26962e6299` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:05.339277 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd5a0c-71b3-4843-9e03-0ebe03cccdd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56765a3804124aa183444b26962e6299)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:05.341535 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd5a0c-71b3-4843-9e03-0ebe03cccdd2 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:05.586237 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-26fd5a0c-71b3-4843-9e03-0ebe03cccdd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56765a3804124aa183444b26962e6299'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:05.600928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-26fd5a0c-71b3-4843-9e03-0ebe03cccdd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56765a3804124aa183444b26962e6299'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:05.714002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-26fd5a0c-71b3-4843-9e03-0ebe03cccdd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56765a3804124aa183444b26962e6299'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:05.714742 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-26fd5a0c-71b3-4843-9e03-0ebe03cccdd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56765a3804124aa183444b26962e6299'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:05.715518 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-26fd5a0c-71b3-4843-9e03-0ebe03cccdd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56765a3804124aa183444b26962e6299'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:05.736726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1090/2203] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:56:05 2022] DELETE /identity/v3/users/56765a3804124aa183444b26962e6299 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:56:05.744570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106dc5b-754a-4393-a790-4d548432298e None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:05.745067 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106dc5b-754a-4393-a790-4d548432298e None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:05.745184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106dc5b-754a-4393-a790-4d548432298e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:05.770382 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-5106dc5b-754a-4393-a790-4d548432298e None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:05.771588 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-5106dc5b-754a-4393-a790-4d548432298e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:05.835623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1114/2204] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:56:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:56:05.845517 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd5a0c-71b3-4843-9e03-0ebe03cccdd2 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:05.847019 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-26fd5a0c-71b3-4843-9e03-0ebe03cccdd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:05.909725 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd5a0c-71b3-4843-9e03-0ebe03cccdd2 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:05.913567 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16e813-f0a7-444e-b41b-53b0ea8daefb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:05.914168 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16e813-f0a7-444e-b41b-53b0ea8daefb service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:05.914455 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16e813-f0a7-444e-b41b-53b0ea8daefb service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:05.914725 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16e813-f0a7-444e-b41b-53b0ea8daefb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:05.923702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb16e813-f0a7-444e-b41b-53b0ea8daefb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:05.925756 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb16e813-f0a7-444e-b41b-53b0ea8daefb service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:05.935616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1091/2205] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:56:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:56:06.413430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106dc5b-754a-4393-a790-4d548432298e None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:06.414445 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-5106dc5b-754a-4393-a790-4d548432298e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:06.430158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106dc5b-754a-4393-a790-4d548432298e None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:06.438930 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e7fa9-78e8-413f-8017-82d404d61b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:06.440235 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e7fa9-78e8-413f-8017-82d404d61b08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:06.440667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e7fa9-78e8-413f-8017-82d404d61b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:06.441013 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e7fa9-78e8-413f-8017-82d404d61b08 admin admin] PATH_INFO: `/v3/projects/7b16fe486c914268b647025b18c7f437` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:06.442511 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e7fa9-78e8-413f-8017-82d404d61b08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b16fe486c914268b647025b18c7f437)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:06.446050 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e7fa9-78e8-413f-8017-82d404d61b08 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:06.593316 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-104e7fa9-78e8-413f-8017-82d404d61b08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b16fe486c914268b647025b18c7f437', 'request_id': 'req-104e7fa9-78e8-413f-8017-82d404d61b08'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:06.675413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-104e7fa9-78e8-413f-8017-82d404d61b08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b16fe486c914268b647025b18c7f437', 'request_id': 'req-104e7fa9-78e8-413f-8017-82d404d61b08'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:06.682040 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-104e7fa9-78e8-413f-8017-82d404d61b08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b16fe486c914268b647025b18c7f437', 'request_id': 'req-104e7fa9-78e8-413f-8017-82d404d61b08'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:06.697857 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-104e7fa9-78e8-413f-8017-82d404d61b08 admin admin] The token cache is being invalidate because project 7b16fe486c914268b647025b18c7f437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:56:06.698151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-104e7fa9-78e8-413f-8017-82d404d61b08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:06.698853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-104e7fa9-78e8-413f-8017-82d404d61b08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:06.700343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1115/2206] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:56:06 2022] DELETE /identity/v3/projects/7b16fe486c914268b647025b18c7f437 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:56:06.704397 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16e813-f0a7-444e-b41b-53b0ea8daefb service neutron] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:06.715382 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16e813-f0a7-444e-b41b-53b0ea8daefb service neutron] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:06.734412 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:06.775630 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8675d0-da0c-41dd-959b-6243d6cccd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:06.776252 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8675d0-da0c-41dd-959b-6243d6cccd4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:06.776569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8675d0-da0c-41dd-959b-6243d6cccd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:06.776847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8675d0-da0c-41dd-959b-6243d6cccd4f admin admin] PATH_INFO: `/v3/users/5d289425b15d4d4c87de65d34c12273b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:06.777981 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8675d0-da0c-41dd-959b-6243d6cccd4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d289425b15d4d4c87de65d34c12273b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:06.779911 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8675d0-da0c-41dd-959b-6243d6cccd4f admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:06.942599 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1b8675d0-da0c-41dd-959b-6243d6cccd4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d289425b15d4d4c87de65d34c12273b'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:06.957636 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1b8675d0-da0c-41dd-959b-6243d6cccd4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d289425b15d4d4c87de65d34c12273b'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:07.029295 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1b8675d0-da0c-41dd-959b-6243d6cccd4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d289425b15d4d4c87de65d34c12273b'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:07.029948 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1b8675d0-da0c-41dd-959b-6243d6cccd4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d289425b15d4d4c87de65d34c12273b'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:07.030529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1b8675d0-da0c-41dd-959b-6243d6cccd4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d289425b15d4d4c87de65d34c12273b'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:07.049878 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1092/2207] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:56:06 2022] DELETE /identity/v3/users/5d289425b15d4d4c87de65d34c12273b => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:56:07.540155 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e7fa9-78e8-413f-8017-82d404d61b08 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:07.542250 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-104e7fa9-78e8-413f-8017-82d404d61b08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:07.557137 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e7fa9-78e8-413f-8017-82d404d61b08 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:07.620252 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73a3f6-bef5-4674-b7f3-9226a8a62ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:07.620928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73a3f6-bef5-4674-b7f3-9226a8a62ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:07.621211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73a3f6-bef5-4674-b7f3-9226a8a62ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:07.621488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73a3f6-bef5-4674-b7f3-9226a8a62ac9 admin admin] PATH_INFO: `/v3/projects/5d131a9aea474f21b0fce6c1729670bb` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:07.622616 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f73a3f6-bef5-4674-b7f3-9226a8a62ac9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d131a9aea474f21b0fce6c1729670bb)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:07.624630 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f73a3f6-bef5-4674-b7f3-9226a8a62ac9 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:07.696355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:07.715881 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2f73a3f6-bef5-4674-b7f3-9226a8a62ac9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d131a9aea474f21b0fce6c1729670bb', 'request_id': 'req-2f73a3f6-bef5-4674-b7f3-9226a8a62ac9'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:07.772244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2f73a3f6-bef5-4674-b7f3-9226a8a62ac9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d131a9aea474f21b0fce6c1729670bb', 'request_id': 'req-2f73a3f6-bef5-4674-b7f3-9226a8a62ac9'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:07.778581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2f73a3f6-bef5-4674-b7f3-9226a8a62ac9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d131a9aea474f21b0fce6c1729670bb', 'request_id': 'req-2f73a3f6-bef5-4674-b7f3-9226a8a62ac9'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:07.791197 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2f73a3f6-bef5-4674-b7f3-9226a8a62ac9 admin admin] The token cache is being invalidate because project 5d131a9aea474f21b0fce6c1729670bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:56:07.791495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2f73a3f6-bef5-4674-b7f3-9226a8a62ac9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:07.792213 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2f73a3f6-bef5-4674-b7f3-9226a8a62ac9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:07.793892 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1116/2208] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:56:07 2022] DELETE /identity/v3/projects/5d131a9aea474f21b0fce6c1729670bb => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:56:07.799484 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de7460-d306-4d9f-8716-1b76b5e30b5b None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:07.799914 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de7460-d306-4d9f-8716-1b76b5e30b5b None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:07.800118 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de7460-d306-4d9f-8716-1b76b5e30b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:07.822659 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-a5de7460-d306-4d9f-8716-1b76b5e30b5b None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:07.823975 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-a5de7460-d306-4d9f-8716-1b76b5e30b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:07.890723 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1093/2209] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:56:07.899420 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab056e-88c7-4a8b-a0c1-7b9c153dd552 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:07.899810 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab056e-88c7-4a8b-a0c1-7b9c153dd552 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:07.900092 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab056e-88c7-4a8b-a0c1-7b9c153dd552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:07.922603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-a4ab056e-88c7-4a8b-a0c1-7b9c153dd552 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:07.923885 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a4ab056e-88c7-4a8b-a0c1-7b9c153dd552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:07.932044 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1117/2210] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:56:07.936559 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de7460-d306-4d9f-8716-1b76b5e30b5b None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:07.946479 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de7460-d306-4d9f-8716-1b76b5e30b5b None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:07.951085 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae0989-0179-4fcf-89d0-1959a4b8364f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:07.951685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae0989-0179-4fcf-89d0-1959a4b8364f admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:07.952049 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae0989-0179-4fcf-89d0-1959a4b8364f admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:07.952271 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae0989-0179-4fcf-89d0-1959a4b8364f admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:07.952953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae0989-0179-4fcf-89d0-1959a4b8364f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:56:07.953254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae0989-0179-4fcf-89d0-1959a4b8364f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:07.955414 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae0989-0179-4fcf-89d0-1959a4b8364f admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:07.967672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1094/2211] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:56:07.972556 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab056e-88c7-4a8b-a0c1-7b9c153dd552 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:07.982078 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab056e-88c7-4a8b-a0c1-7b9c153dd552 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:07.986761 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9376634-f317-4ef8-97c0-4d055479ac43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:07.987361 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9376634-f317-4ef8-97c0-4d055479ac43 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:07.987635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9376634-f317-4ef8-97c0-4d055479ac43 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:07.987901 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9376634-f317-4ef8-97c0-4d055479ac43 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:07.988568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9376634-f317-4ef8-97c0-4d055479ac43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:07.990540 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9376634-f317-4ef8-97c0-4d055479ac43 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:08.071813 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1118/2212] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:56:07 2022] POST /identity/v3/projects => generated 369 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:56:08.076021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae0989-0179-4fcf-89d0-1959a4b8364f admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:08.095119 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae0989-0179-4fcf-89d0-1959a4b8364f admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:08.126144 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:08.157797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef47b49b-2910-48f0-8ebd-a8ac4287a673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:08.159229 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47b49b-2910-48f0-8ebd-a8ac4287a673 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:08.159513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47b49b-2910-48f0-8ebd-a8ac4287a673 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:08.159791 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47b49b-2910-48f0-8ebd-a8ac4287a673 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:08.160434 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef47b49b-2910-48f0-8ebd-a8ac4287a673 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:08.162648 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef47b49b-2910-48f0-8ebd-a8ac4287a673 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:08.253615 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1095/2213] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:56:08 2022] POST /identity/v3/users => generated 285 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:56:08.257869 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9376634-f317-4ef8-97c0-4d055479ac43 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:08.269423 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9376634-f317-4ef8-97c0-4d055479ac43 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:08.274211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363fa10-6a78-483f-9aff-7823d9b6aec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:08.274849 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363fa10-6a78-483f-9aff-7823d9b6aec2 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:08.275136 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363fa10-6a78-483f-9aff-7823d9b6aec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:08.275406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363fa10-6a78-483f-9aff-7823d9b6aec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:08.276052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a363fa10-6a78-483f-9aff-7823d9b6aec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:08.277703 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a363fa10-6a78-483f-9aff-7823d9b6aec2 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:08.289350 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1119/2214] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:56:08.293424 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef47b49b-2910-48f0-8ebd-a8ac4287a673 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:08.304558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef47b49b-2910-48f0-8ebd-a8ac4287a673 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:08.309230 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c1652-8940-4c8b-aa5e-f5abdc4ae653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:08.309940 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c1652-8940-4c8b-aa5e-f5abdc4ae653 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:08.310243 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c1652-8940-4c8b-aa5e-f5abdc4ae653 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:08.310493 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c1652-8940-4c8b-aa5e-f5abdc4ae653 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:08.311133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c1652-8940-4c8b-aa5e-f5abdc4ae653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:08.312748 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c1652-8940-4c8b-aa5e-f5abdc4ae653 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:08.323415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1096/2215] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:56:08.327633 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363fa10-6a78-483f-9aff-7823d9b6aec2 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:08.337535 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363fa10-6a78-483f-9aff-7823d9b6aec2 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:08.342075 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a557ab85-a8b6-4ef1-9890-085147ad23fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:08.342726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a557ab85-a8b6-4ef1-9890-085147ad23fe admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:08.343013 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a557ab85-a8b6-4ef1-9890-085147ad23fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:08.343284 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a557ab85-a8b6-4ef1-9890-085147ad23fe admin admin] PATH_INFO: `/v3/projects/3fb04a1d266046c7acbe3e6a64ad4e61/users/85272d72d7c14880a3bdd781ea48d9cb/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:08.358708 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a557ab85-a8b6-4ef1-9890-085147ad23fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fb04a1d266046c7acbe3e6a64ad4e61, user_id=85272d72d7c14880a3bdd781ea48d9cb, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:08.361963 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a557ab85-a8b6-4ef1-9890-085147ad23fe admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:08.441608 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1120/2216] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:56:08 2022] PUT /identity/v3/projects/3fb04a1d266046c7acbe3e6a64ad4e61/users/85272d72d7c14880a3bdd781ea48d9cb/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:56:08.453152 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dfffc9-b836-4a27-bca6-5f87472c8b6e None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:08.453443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dfffc9-b836-4a27-bca6-5f87472c8b6e None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:08.453718 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dfffc9-b836-4a27-bca6-5f87472c8b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:08.483769 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-88dfffc9-b836-4a27-bca6-5f87472c8b6e None None] MFA Rules not processed for user `85272d72d7c14880a3bdd781ea48d9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:08.485056 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-88dfffc9-b836-4a27-bca6-5f87472c8b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:08.552602 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1097/2217] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:56:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:56:08.557076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a557ab85-a8b6-4ef1-9890-085147ad23fe admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:08.620880 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a557ab85-a8b6-4ef1-9890-085147ad23fe admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:08.625527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b3972-a017-47f6-a738-5b266cf559ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:08.626133 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b3972-a017-47f6-a738-5b266cf559ff admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:08.626425 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b3972-a017-47f6-a738-5b266cf559ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:08.626814 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b3972-a017-47f6-a738-5b266cf559ff admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:08.627360 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b3972-a017-47f6-a738-5b266cf559ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:08.629384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b3972-a017-47f6-a738-5b266cf559ff admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:08.735038 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1121/2218] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:56:08 2022] POST /identity/v3/projects => generated 369 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:56:08.740679 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dfffc9-b836-4a27-bca6-5f87472c8b6e None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:08.752503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dfffc9-b836-4a27-bca6-5f87472c8b6e None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:08.815044 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d4e02-255f-4034-89ac-54b5e76f4982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:08.816418 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d4e02-255f-4034-89ac-54b5e76f4982 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:08.816848 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d4e02-255f-4034-89ac-54b5e76f4982 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:08.817131 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d4e02-255f-4034-89ac-54b5e76f4982 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:08.817805 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d4e02-255f-4034-89ac-54b5e76f4982 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:08.819764 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d4e02-255f-4034-89ac-54b5e76f4982 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:08.926756 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1098/2219] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:56:08 2022] POST /identity/v3/users => generated 291 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:56:08.930848 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b3972-a017-47f6-a738-5b266cf559ff admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:08.945995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b3972-a017-47f6-a738-5b266cf559ff admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:08.950701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f10031-2526-44c7-ba2e-244503d1fadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:08.951310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f10031-2526-44c7-ba2e-244503d1fadf admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:08.951587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f10031-2526-44c7-ba2e-244503d1fadf admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:08.951858 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f10031-2526-44c7-ba2e-244503d1fadf admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:08.952553 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f10031-2526-44c7-ba2e-244503d1fadf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:08.954165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f10031-2526-44c7-ba2e-244503d1fadf admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:08.966057 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1122/2220] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:56:08.971704 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d4e02-255f-4034-89ac-54b5e76f4982 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:08.982264 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d4e02-255f-4034-89ac-54b5e76f4982 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:08.986998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8116df-4924-48ac-b8a6-dcc91f877275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:08.987770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8116df-4924-48ac-b8a6-dcc91f877275 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:08.987893 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8116df-4924-48ac-b8a6-dcc91f877275 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:08.988134 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8116df-4924-48ac-b8a6-dcc91f877275 admin admin] PATH_INFO: `/v3/projects/c53e2f3f75d9473cbdf2b9b5de703807/users/90d0b8d84a97434dbdb5ac4b855e9fa9/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:09.006138 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8116df-4924-48ac-b8a6-dcc91f877275 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c53e2f3f75d9473cbdf2b9b5de703807, user_id=90d0b8d84a97434dbdb5ac4b855e9fa9, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:09.008881 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8116df-4924-48ac-b8a6-dcc91f877275 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:09.109709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1099/2221] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:56:08 2022] PUT /identity/v3/projects/c53e2f3f75d9473cbdf2b9b5de703807/users/90d0b8d84a97434dbdb5ac4b855e9fa9/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:56:09.114875 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f10031-2526-44c7-ba2e-244503d1fadf admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:09.126002 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f10031-2526-44c7-ba2e-244503d1fadf admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:09.146101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:09.187320 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba4739-ce17-424c-b627-8861496ed83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:09.187947 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba4739-ce17-424c-b627-8861496ed83d admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:09.189587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba4739-ce17-424c-b627-8861496ed83d admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:09.189587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba4739-ce17-424c-b627-8861496ed83d admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:09.189587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba4739-ce17-424c-b627-8861496ed83d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:09.191091 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba4739-ce17-424c-b627-8861496ed83d admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:09.201284 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1123/2222] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:56:09.205844 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8116df-4924-48ac-b8a6-dcc91f877275 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:09.212766 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:09.218271 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8116df-4924-48ac-b8a6-dcc91f877275 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:09.223773 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b7bfa-f6b7-4e99-b1cb-beb53f442bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:09.224540 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b7bfa-f6b7-4e99-b1cb-beb53f442bf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:09.224889 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b7bfa-f6b7-4e99-b1cb-beb53f442bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:09.225227 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b7bfa-f6b7-4e99-b1cb-beb53f442bf8 admin admin] PATH_INFO: `/v3/projects/c53e2f3f75d9473cbdf2b9b5de703807/users/90d0b8d84a97434dbdb5ac4b855e9fa9/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:09.227261 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b7bfa-f6b7-4e99-b1cb-beb53f442bf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c53e2f3f75d9473cbdf2b9b5de703807, user_id=90d0b8d84a97434dbdb5ac4b855e9fa9, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:09.230048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b7bfa-f6b7-4e99-b1cb-beb53f442bf8 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:09.242706 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1100/2223] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:56:09 2022] PUT /identity/v3/projects/c53e2f3f75d9473cbdf2b9b5de703807/users/90d0b8d84a97434dbdb5ac4b855e9fa9/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:56:09.253153 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72030c-14fb-46bb-a692-3ceec4466fd2 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:09.253442 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72030c-14fb-46bb-a692-3ceec4466fd2 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:09.253718 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72030c-14fb-46bb-a692-3ceec4466fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:09.286117 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-dd72030c-14fb-46bb-a692-3ceec4466fd2 None None] MFA Rules not processed for user `90d0b8d84a97434dbdb5ac4b855e9fa9`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:09.287587 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-dd72030c-14fb-46bb-a692-3ceec4466fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:09.371803 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1124/2224] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:56:09 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:56:09.383387 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b7bfa-f6b7-4e99-b1cb-beb53f442bf8 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:09.385451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-096b7bfa-f6b7-4e99-b1cb-beb53f442bf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:09.445105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b7bfa-f6b7-4e99-b1cb-beb53f442bf8 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:09.448792 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4158a-0176-4bbf-a9cd-9f332df3896f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:09.449398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4158a-0176-4bbf-a9cd-9f332df3896f service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:09.449674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4158a-0176-4bbf-a9cd-9f332df3896f service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:09.449944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4158a-0176-4bbf-a9cd-9f332df3896f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:09.460301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4158a-0176-4bbf-a9cd-9f332df3896f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:09.461796 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4158a-0176-4bbf-a9cd-9f332df3896f service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:09.472047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1101/2225] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:56:09 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:56:09.591003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72030c-14fb-46bb-a692-3ceec4466fd2 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:09.608506 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72030c-14fb-46bb-a692-3ceec4466fd2 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:09.613552 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ba12f-698f-4250-b83d-33664748324e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:09.614569 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ba12f-698f-4250-b83d-33664748324e service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:09.615064 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ba12f-698f-4250-b83d-33664748324e service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:09.615531 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ba12f-698f-4250-b83d-33664748324e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:09.681679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ba12f-698f-4250-b83d-33664748324e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:09.683215 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ba12f-698f-4250-b83d-33664748324e service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:09.692552 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1125/2226] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:56:09 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:56:15.102365 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4158a-0176-4bbf-a9cd-9f332df3896f service cinder] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:15.173096 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4158a-0176-4bbf-a9cd-9f332df3896f service cinder] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:15.178834 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586fa12-f3e6-4632-8a06-0ac278baa750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:15.179534 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586fa12-f3e6-4632-8a06-0ac278baa750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:15.179827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586fa12-f3e6-4632-8a06-0ac278baa750 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:15.180105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586fa12-f3e6-4632-8a06-0ac278baa750 admin admin] PATH_INFO: `/v3/users/85272d72d7c14880a3bdd781ea48d9cb` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:15.181305 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b586fa12-f3e6-4632-8a06-0ac278baa750 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85272d72d7c14880a3bdd781ea48d9cb)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:15.183302 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b586fa12-f3e6-4632-8a06-0ac278baa750 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:15.368528 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-b586fa12-f3e6-4632-8a06-0ac278baa750 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85272d72d7c14880a3bdd781ea48d9cb'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:15.381000 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-b586fa12-f3e6-4632-8a06-0ac278baa750 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85272d72d7c14880a3bdd781ea48d9cb'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:15.475247 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-b586fa12-f3e6-4632-8a06-0ac278baa750 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85272d72d7c14880a3bdd781ea48d9cb'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:15.475962 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-b586fa12-f3e6-4632-8a06-0ac278baa750 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85272d72d7c14880a3bdd781ea48d9cb'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:15.476632 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-b586fa12-f3e6-4632-8a06-0ac278baa750 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85272d72d7c14880a3bdd781ea48d9cb'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:15.490922 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1102/2227] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:56:15 2022] DELETE /identity/v3/users/85272d72d7c14880a3bdd781ea48d9cb => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:56:15.495470 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fa703-3b22-42e1-8d3e-5f378e94d850 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:15.495971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fa703-3b22-42e1-8d3e-5f378e94d850 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:15.496066 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fa703-3b22-42e1-8d3e-5f378e94d850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:15.516026 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-825fa703-3b22-42e1-8d3e-5f378e94d850 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:15.517273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-825fa703-3b22-42e1-8d3e-5f378e94d850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:15.556887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:15.575354 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1126/2228] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:56:15.585572 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586fa12-f3e6-4632-8a06-0ac278baa750 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:15.586973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-b586fa12-f3e6-4632-8a06-0ac278baa750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:15.600635 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:15.645455 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586fa12-f3e6-4632-8a06-0ac278baa750 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:15.649126 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7cad79-5049-4708-8d1b-987ca61d27bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:15.649744 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cad79-5049-4708-8d1b-987ca61d27bc service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:15.650004 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cad79-5049-4708-8d1b-987ca61d27bc service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:15.650275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cad79-5049-4708-8d1b-987ca61d27bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:15.659554 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7cad79-5049-4708-8d1b-987ca61d27bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:15.669887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7cad79-5049-4708-8d1b-987ca61d27bc service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:15.671362 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1103/2229] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:56:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:56:16.179283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825fa703-3b22-42e1-8d3e-5f378e94d850 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:16.180762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-825fa703-3b22-42e1-8d3e-5f378e94d850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:16.193574 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825fa703-3b22-42e1-8d3e-5f378e94d850 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:16.198740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394aa826-e6e9-49f7-813d-247ea1d182e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:16.199407 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394aa826-e6e9-49f7-813d-247ea1d182e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:16.199732 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394aa826-e6e9-49f7-813d-247ea1d182e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:16.199959 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394aa826-e6e9-49f7-813d-247ea1d182e5 admin admin] PATH_INFO: `/v3/projects/3fb04a1d266046c7acbe3e6a64ad4e61` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:16.201108 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394aa826-e6e9-49f7-813d-247ea1d182e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fb04a1d266046c7acbe3e6a64ad4e61)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:16.203034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394aa826-e6e9-49f7-813d-247ea1d182e5 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:16.315549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-394aa826-e6e9-49f7-813d-247ea1d182e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fb04a1d266046c7acbe3e6a64ad4e61', 'request_id': 'req-394aa826-e6e9-49f7-813d-247ea1d182e5'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:16.389265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-394aa826-e6e9-49f7-813d-247ea1d182e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fb04a1d266046c7acbe3e6a64ad4e61', 'request_id': 'req-394aa826-e6e9-49f7-813d-247ea1d182e5'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:16.395786 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-394aa826-e6e9-49f7-813d-247ea1d182e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fb04a1d266046c7acbe3e6a64ad4e61', 'request_id': 'req-394aa826-e6e9-49f7-813d-247ea1d182e5'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:16.408371 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-394aa826-e6e9-49f7-813d-247ea1d182e5 admin admin] The token cache is being invalidate because project 3fb04a1d266046c7acbe3e6a64ad4e61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:56:16.408731 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-394aa826-e6e9-49f7-813d-247ea1d182e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:16.409421 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-394aa826-e6e9-49f7-813d-247ea1d182e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:16.411784 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1127/2230] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:56:16 2022] DELETE /identity/v3/projects/3fb04a1d266046c7acbe3e6a64ad4e61 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:56:16.415429 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7cad79-5049-4708-8d1b-987ca61d27bc service neutron] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:16.425902 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7cad79-5049-4708-8d1b-987ca61d27bc service neutron] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:16.496295 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55fa053-cd40-4f5c-affc-57bcf1c6510b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:16.496295 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fa053-cd40-4f5c-affc-57bcf1c6510b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:16.496295 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fa053-cd40-4f5c-affc-57bcf1c6510b admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:16.496295 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fa053-cd40-4f5c-affc-57bcf1c6510b admin admin] PATH_INFO: `/v3/users/90d0b8d84a97434dbdb5ac4b855e9fa9` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:16.496747 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55fa053-cd40-4f5c-affc-57bcf1c6510b admin admin] RBAC: Authorizing `identity:delete_user(user_id=90d0b8d84a97434dbdb5ac4b855e9fa9)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:16.496950 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55fa053-cd40-4f5c-affc-57bcf1c6510b admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:16.774282 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f55fa053-cd40-4f5c-affc-57bcf1c6510b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90d0b8d84a97434dbdb5ac4b855e9fa9'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:16.786990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f55fa053-cd40-4f5c-affc-57bcf1c6510b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90d0b8d84a97434dbdb5ac4b855e9fa9'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:16.891613 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f55fa053-cd40-4f5c-affc-57bcf1c6510b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90d0b8d84a97434dbdb5ac4b855e9fa9'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:16.892811 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f55fa053-cd40-4f5c-affc-57bcf1c6510b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90d0b8d84a97434dbdb5ac4b855e9fa9'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:16.893483 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f55fa053-cd40-4f5c-affc-57bcf1c6510b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90d0b8d84a97434dbdb5ac4b855e9fa9'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:16.906480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1104/2231] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:56:16 2022] DELETE /identity/v3/users/90d0b8d84a97434dbdb5ac4b855e9fa9 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:56:17.334403 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394aa826-e6e9-49f7-813d-247ea1d182e5 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:17.336499 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-394aa826-e6e9-49f7-813d-247ea1d182e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:17.354083 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394aa826-e6e9-49f7-813d-247ea1d182e5 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:17.422795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3ebbd-fde3-4c16-8975-4251b3202292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:17.423458 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3ebbd-fde3-4c16-8975-4251b3202292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:17.423714 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3ebbd-fde3-4c16-8975-4251b3202292 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:17.423985 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3ebbd-fde3-4c16-8975-4251b3202292 admin admin] PATH_INFO: `/v3/projects/c53e2f3f75d9473cbdf2b9b5de703807` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:17.425137 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3ebbd-fde3-4c16-8975-4251b3202292 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c53e2f3f75d9473cbdf2b9b5de703807)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:17.427045 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3ebbd-fde3-4c16-8975-4251b3202292 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:17.574808 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-72e3ebbd-fde3-4c16-8975-4251b3202292 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c53e2f3f75d9473cbdf2b9b5de703807', 'request_id': 'req-72e3ebbd-fde3-4c16-8975-4251b3202292'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:17.692210 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-72e3ebbd-fde3-4c16-8975-4251b3202292 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c53e2f3f75d9473cbdf2b9b5de703807', 'request_id': 'req-72e3ebbd-fde3-4c16-8975-4251b3202292'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:17.699758 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-72e3ebbd-fde3-4c16-8975-4251b3202292 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c53e2f3f75d9473cbdf2b9b5de703807', 'request_id': 'req-72e3ebbd-fde3-4c16-8975-4251b3202292'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:17.707916 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:17.713135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-72e3ebbd-fde3-4c16-8975-4251b3202292 admin admin] The token cache is being invalidate because project c53e2f3f75d9473cbdf2b9b5de703807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:56:17.713522 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-72e3ebbd-fde3-4c16-8975-4251b3202292 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:17.714206 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-72e3ebbd-fde3-4c16-8975-4251b3202292 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:17.715766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1128/2232] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:56:17 2022] DELETE /identity/v3/projects/c53e2f3f75d9473cbdf2b9b5de703807 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:56:17.721039 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0b050-02a4-451d-b977-59e17f84c301 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:17.721520 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0b050-02a4-451d-b977-59e17f84c301 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:17.721635 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0b050-02a4-451d-b977-59e17f84c301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:17.742898 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-7af0b050-02a4-451d-b977-59e17f84c301 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:17.744104 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-7af0b050-02a4-451d-b977-59e17f84c301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:17.780817 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:17.812590 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1105/2233] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:56:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:56:17.822931 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6178169-3f1a-42b2-af42-e249c018110a None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:17.823159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6178169-3f1a-42b2-af42-e249c018110a None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:17.823431 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6178169-3f1a-42b2-af42-e249c018110a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:17.843106 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-c6178169-3f1a-42b2-af42-e249c018110a None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:17.844311 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-c6178169-3f1a-42b2-af42-e249c018110a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:17.852919 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1129/2234] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:56:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:56:17.857501 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0b050-02a4-451d-b977-59e17f84c301 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:17.867681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0b050-02a4-451d-b977-59e17f84c301 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:17.872523 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b582076-7fd5-4765-85e7-ef6eec33772c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:17.873102 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b582076-7fd5-4765-85e7-ef6eec33772c admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:17.873382 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b582076-7fd5-4765-85e7-ef6eec33772c admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:17.873699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b582076-7fd5-4765-85e7-ef6eec33772c admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:17.874331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b582076-7fd5-4765-85e7-ef6eec33772c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:56:17.874636 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b582076-7fd5-4765-85e7-ef6eec33772c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:17.876430 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b582076-7fd5-4765-85e7-ef6eec33772c admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:17.888509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1106/2235] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:56:17.893782 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6178169-3f1a-42b2-af42-e249c018110a None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:17.902995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6178169-3f1a-42b2-af42-e249c018110a None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:17.907679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78739d15-281c-41a8-8bd6-0282399fdf4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:17.908846 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78739d15-281c-41a8-8bd6-0282399fdf4b admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:17.909185 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78739d15-281c-41a8-8bd6-0282399fdf4b admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:17.909463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78739d15-281c-41a8-8bd6-0282399fdf4b admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:17.910096 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78739d15-281c-41a8-8bd6-0282399fdf4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:17.912027 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78739d15-281c-41a8-8bd6-0282399fdf4b admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:18.053300 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1130/2236] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:56:17 2022] POST /identity/v3/projects => generated 387 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:56:18.057618 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b582076-7fd5-4765-85e7-ef6eec33772c admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:18.071327 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b582076-7fd5-4765-85e7-ef6eec33772c admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:18.144275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb23221-75b5-4a7e-8d8b-7a16fd991a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:18.144275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb23221-75b5-4a7e-8d8b-7a16fd991a45 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:18.144662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb23221-75b5-4a7e-8d8b-7a16fd991a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:18.144888 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb23221-75b5-4a7e-8d8b-7a16fd991a45 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:18.145521 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb23221-75b5-4a7e-8d8b-7a16fd991a45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:18.147593 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb23221-75b5-4a7e-8d8b-7a16fd991a45 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:18.259482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1107/2237] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:56:18 2022] POST /identity/v3/users => generated 294 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:56:18.263895 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78739d15-281c-41a8-8bd6-0282399fdf4b admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:18.274545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78739d15-281c-41a8-8bd6-0282399fdf4b admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:18.279352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2c5f0-14ef-43ba-bf10-b821475afe4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:18.279545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2c5f0-14ef-43ba-bf10-b821475afe4f admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:18.279824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2c5f0-14ef-43ba-bf10-b821475afe4f admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:18.280094 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2c5f0-14ef-43ba-bf10-b821475afe4f admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:18.280775 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2c5f0-14ef-43ba-bf10-b821475afe4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:18.282355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2c5f0-14ef-43ba-bf10-b821475afe4f admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:18.293045 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1131/2238] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:56:18.297014 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb23221-75b5-4a7e-8d8b-7a16fd991a45 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:18.306834 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb23221-75b5-4a7e-8d8b-7a16fd991a45 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:18.313766 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35386d-741e-466c-8b9e-71febb11b296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:18.313766 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35386d-741e-466c-8b9e-71febb11b296 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:18.313766 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35386d-741e-466c-8b9e-71febb11b296 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:18.313766 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35386d-741e-466c-8b9e-71febb11b296 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:18.314678 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35386d-741e-466c-8b9e-71febb11b296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:18.315206 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35386d-741e-466c-8b9e-71febb11b296 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:18.325601 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1108/2239] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:56:18.329610 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2c5f0-14ef-43ba-bf10-b821475afe4f admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:18.339763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2c5f0-14ef-43ba-bf10-b821475afe4f admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:18.344217 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe435fac-c7f1-4503-aa22-050587213208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:18.344899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe435fac-c7f1-4503-aa22-050587213208 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:18.345205 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe435fac-c7f1-4503-aa22-050587213208 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:18.345479 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe435fac-c7f1-4503-aa22-050587213208 admin admin] PATH_INFO: `/v3/projects/40a2b56f30e44eeeb3dcb11dced91a91/users/8b1ff1e536214081aeb4362c8c845df7/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:18.359689 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe435fac-c7f1-4503-aa22-050587213208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40a2b56f30e44eeeb3dcb11dced91a91, user_id=8b1ff1e536214081aeb4362c8c845df7, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:18.362766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe435fac-c7f1-4503-aa22-050587213208 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:18.424748 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1132/2240] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:56:18 2022] PUT /identity/v3/projects/40a2b56f30e44eeeb3dcb11dced91a91/users/8b1ff1e536214081aeb4362c8c845df7/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:56:18.436252 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ff1c7-6049-4c15-8d21-bbef39496923 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:18.437298 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ff1c7-6049-4c15-8d21-bbef39496923 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:18.437654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ff1c7-6049-4c15-8d21-bbef39496923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:18.467966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-792ff1c7-6049-4c15-8d21-bbef39496923 None None] MFA Rules not processed for user `8b1ff1e536214081aeb4362c8c845df7`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:18.469321 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-792ff1c7-6049-4c15-8d21-bbef39496923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:18.539199 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1109/2241] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:56:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:56:18.543715 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe435fac-c7f1-4503-aa22-050587213208 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:18.623807 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe435fac-c7f1-4503-aa22-050587213208 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:18.629298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c0314-7a2b-4496-915e-b598d915a766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:18.629904 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c0314-7a2b-4496-915e-b598d915a766 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:18.630186 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c0314-7a2b-4496-915e-b598d915a766 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:18.630487 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c0314-7a2b-4496-915e-b598d915a766 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:18.631126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c0314-7a2b-4496-915e-b598d915a766 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:18.633082 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c0314-7a2b-4496-915e-b598d915a766 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:18.777754 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1133/2242] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:56:18 2022] POST /identity/v3/projects => generated 385 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:56:18.781946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ff1c7-6049-4c15-8d21-bbef39496923 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:18.797969 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ff1c7-6049-4c15-8d21-bbef39496923 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:18.834279 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:18.922899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a1189-3de0-427f-b0d1-40868882fdd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:18.923788 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a1189-3de0-427f-b0d1-40868882fdd6 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:18.924122 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a1189-3de0-427f-b0d1-40868882fdd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:18.924402 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a1189-3de0-427f-b0d1-40868882fdd6 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:18.925183 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5a1189-3de0-427f-b0d1-40868882fdd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:18.927125 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5a1189-3de0-427f-b0d1-40868882fdd6 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:19.036607 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1110/2243] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:56:18 2022] POST /identity/v3/users => generated 299 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:56:19.038727 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c0314-7a2b-4496-915e-b598d915a766 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:19.055463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c0314-7a2b-4496-915e-b598d915a766 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:19.065747 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b5b1f-aa55-49d9-bd80-86ec393a106b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:19.066663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b5b1f-aa55-49d9-bd80-86ec393a106b admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:19.067125 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b5b1f-aa55-49d9-bd80-86ec393a106b admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:19.068081 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b5b1f-aa55-49d9-bd80-86ec393a106b admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:19.069106 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b5b1f-aa55-49d9-bd80-86ec393a106b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:19.072771 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b5b1f-aa55-49d9-bd80-86ec393a106b admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:19.095341 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1134/2244] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:56:19.096120 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a1189-3de0-427f-b0d1-40868882fdd6 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:19.107374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a1189-3de0-427f-b0d1-40868882fdd6 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:19.136754 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76d041-561b-403a-8752-e1dc7773e041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:19.137693 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76d041-561b-403a-8752-e1dc7773e041 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:19.138496 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76d041-561b-403a-8752-e1dc7773e041 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:19.143618 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76d041-561b-403a-8752-e1dc7773e041 admin admin] PATH_INFO: `/v3/projects/2fdb9bff842f421baa9660fde546c405/users/0ee92728522e4f60a860805da8e2cdc3/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:19.179962 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a76d041-561b-403a-8752-e1dc7773e041 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fdb9bff842f421baa9660fde546c405, user_id=0ee92728522e4f60a860805da8e2cdc3, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:19.210291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a76d041-561b-403a-8752-e1dc7773e041 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:19.339215 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1111/2245] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:56:19 2022] PUT /identity/v3/projects/2fdb9bff842f421baa9660fde546c405/users/0ee92728522e4f60a860805da8e2cdc3/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:56:19.343701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b5b1f-aa55-49d9-bd80-86ec393a106b admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:19.362741 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b5b1f-aa55-49d9-bd80-86ec393a106b admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:19.487774 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8136c-f82f-4460-9e64-92bf96727443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:19.488589 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8136c-f82f-4460-9e64-92bf96727443 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:19.488868 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8136c-f82f-4460-9e64-92bf96727443 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:19.489434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8136c-f82f-4460-9e64-92bf96727443 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:19.490136 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e8136c-f82f-4460-9e64-92bf96727443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:19.491959 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e8136c-f82f-4460-9e64-92bf96727443 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:19.504760 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1135/2246] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 00:56:19.508891 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76d041-561b-403a-8752-e1dc7773e041 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:19.522175 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76d041-561b-403a-8752-e1dc7773e041 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:19.527658 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ba014-4e1f-4786-895c-bff0b9e1e619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:19.528785 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ba014-4e1f-4786-895c-bff0b9e1e619 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:19.529101 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ba014-4e1f-4786-895c-bff0b9e1e619 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:19.529374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ba014-4e1f-4786-895c-bff0b9e1e619 admin admin] PATH_INFO: `/v3/projects/2fdb9bff842f421baa9660fde546c405/users/0ee92728522e4f60a860805da8e2cdc3/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:19.531800 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ba014-4e1f-4786-895c-bff0b9e1e619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fdb9bff842f421baa9660fde546c405, user_id=0ee92728522e4f60a860805da8e2cdc3, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:19.535172 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ba014-4e1f-4786-895c-bff0b9e1e619 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:19.546402 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1112/2247] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:56:19 2022] PUT /identity/v3/projects/2fdb9bff842f421baa9660fde546c405/users/0ee92728522e4f60a860805da8e2cdc3/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:56:19.560356 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07748200-77dd-46dc-a063-948f1df793df None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:19.560713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07748200-77dd-46dc-a063-948f1df793df None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:19.560993 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07748200-77dd-46dc-a063-948f1df793df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:19.569010 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:19.590031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-07748200-77dd-46dc-a063-948f1df793df None None] MFA Rules not processed for user `0ee92728522e4f60a860805da8e2cdc3`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:19.591251 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-07748200-77dd-46dc-a063-948f1df793df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:19.665635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1136/2248] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:56:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:56:19.679908 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ba014-4e1f-4786-895c-bff0b9e1e619 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:19.682023 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-912ba014-4e1f-4786-895c-bff0b9e1e619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:19.746360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ba014-4e1f-4786-895c-bff0b9e1e619 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:19.750182 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e34e01-0f18-4879-844f-6cc9bdcdc74b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:19.750793 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e34e01-0f18-4879-844f-6cc9bdcdc74b service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:19.752155 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e34e01-0f18-4879-844f-6cc9bdcdc74b service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:19.752155 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e34e01-0f18-4879-844f-6cc9bdcdc74b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:19.761337 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e34e01-0f18-4879-844f-6cc9bdcdc74b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:19.762772 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e34e01-0f18-4879-844f-6cc9bdcdc74b service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:19.772497 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1113/2249] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:56:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:56:19.851126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07748200-77dd-46dc-a063-948f1df793df None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:19.914249 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07748200-77dd-46dc-a063-948f1df793df None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:19.918959 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76f73e-46cc-45cc-bc59-bc88027911a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:19.919602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76f73e-46cc-45cc-bc59-bc88027911a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:19.919879 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76f73e-46cc-45cc-bc59-bc88027911a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:19.920154 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76f73e-46cc-45cc-bc59-bc88027911a1 admin admin] PATH_INFO: `/v3/users/8b1ff1e536214081aeb4362c8c845df7` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:19.921335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76f73e-46cc-45cc-bc59-bc88027911a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b1ff1e536214081aeb4362c8c845df7)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:19.923271 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76f73e-46cc-45cc-bc59-bc88027911a1 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:20.143197 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-7e76f73e-46cc-45cc-bc59-bc88027911a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b1ff1e536214081aeb4362c8c845df7'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:20.143944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-7e76f73e-46cc-45cc-bc59-bc88027911a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b1ff1e536214081aeb4362c8c845df7'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:20.160506 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-7e76f73e-46cc-45cc-bc59-bc88027911a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b1ff1e536214081aeb4362c8c845df7'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:20.240621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-7e76f73e-46cc-45cc-bc59-bc88027911a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b1ff1e536214081aeb4362c8c845df7'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:20.241477 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-7e76f73e-46cc-45cc-bc59-bc88027911a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b1ff1e536214081aeb4362c8c845df7'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:20.256136 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1137/2250] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:56:19 2022] DELETE /identity/v3/users/8b1ff1e536214081aeb4362c8c845df7 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:56:20.260823 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a282e76-b082-481d-86db-7eb4fb8bc81f None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:20.261396 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a282e76-b082-481d-86db-7eb4fb8bc81f None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:20.261515 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a282e76-b082-481d-86db-7eb4fb8bc81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:20.281192 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-4a282e76-b082-481d-86db-7eb4fb8bc81f None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:20.282389 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-4a282e76-b082-481d-86db-7eb4fb8bc81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:20.312608 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76f73e-46cc-45cc-bc59-bc88027911a1 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:20.314517 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-7e76f73e-46cc-45cc-bc59-bc88027911a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:20.326160 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:20.349712 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1114/2251] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:56:20.359551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a282e76-b082-481d-86db-7eb4fb8bc81f None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:20.361494 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-4a282e76-b082-481d-86db-7eb4fb8bc81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:20.387328 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76f73e-46cc-45cc-bc59-bc88027911a1 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:20.388713 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:20.392298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0548958d-bf6c-41ae-bfa7-956f084524ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:20.392975 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548958d-bf6c-41ae-bfa7-956f084524ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:20.393256 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548958d-bf6c-41ae-bfa7-956f084524ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:20.393551 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548958d-bf6c-41ae-bfa7-956f084524ce admin admin] PATH_INFO: `/v3/users/d959f97aff1e4cf0b6b76d7b4ad9705f` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:20.394665 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0548958d-bf6c-41ae-bfa7-956f084524ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=d959f97aff1e4cf0b6b76d7b4ad9705f)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:20.396647 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0548958d-bf6c-41ae-bfa7-956f084524ce admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:20.421448 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a282e76-b082-481d-86db-7eb4fb8bc81f None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:20.425866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c5dd8-5a60-4fa4-bf0b-ce2fb891b224 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:20.426468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c5dd8-5a60-4fa4-bf0b-ce2fb891b224 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:20.426744 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c5dd8-5a60-4fa4-bf0b-ce2fb891b224 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:20.427014 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c5dd8-5a60-4fa4-bf0b-ce2fb891b224 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:20.442303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c5dd8-5a60-4fa4-bf0b-ce2fb891b224 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:20.445300 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c5dd8-5a60-4fa4-bf0b-ce2fb891b224 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:20.456506 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1115/2252] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:56:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:56:20.632335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-0548958d-bf6c-41ae-bfa7-956f084524ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd959f97aff1e4cf0b6b76d7b4ad9705f'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:20.633023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-0548958d-bf6c-41ae-bfa7-956f084524ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd959f97aff1e4cf0b6b76d7b4ad9705f'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:20.645372 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-0548958d-bf6c-41ae-bfa7-956f084524ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd959f97aff1e4cf0b6b76d7b4ad9705f'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:20.789660 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-0548958d-bf6c-41ae-bfa7-956f084524ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd959f97aff1e4cf0b6b76d7b4ad9705f'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:20.790856 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-0548958d-bf6c-41ae-bfa7-956f084524ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd959f97aff1e4cf0b6b76d7b4ad9705f'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:20.819275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1138/2253] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:56:20 2022] DELETE /identity/v3/users/d959f97aff1e4cf0b6b76d7b4ad9705f => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:56:20.823756 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187ebab-1eed-4eca-9d43-e694d3c2eacb None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:20.824039 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187ebab-1eed-4eca-9d43-e694d3c2eacb None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:20.824723 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187ebab-1eed-4eca-9d43-e694d3c2eacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:20.843487 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-8187ebab-1eed-4eca-9d43-e694d3c2eacb None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:20.844761 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-8187ebab-1eed-4eca-9d43-e694d3c2eacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:20.919021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1116/2254] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:56:20.928832 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0548958d-bf6c-41ae-bfa7-956f084524ce admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:20.930241 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-0548958d-bf6c-41ae-bfa7-956f084524ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:20.993960 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0548958d-bf6c-41ae-bfa7-956f084524ce admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:20.994806 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a69551-1082-4a28-a681-98c7f18b1ef6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:20.995420 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a69551-1082-4a28-a681-98c7f18b1ef6 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:20.995721 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a69551-1082-4a28-a681-98c7f18b1ef6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:20.995994 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a69551-1082-4a28-a681-98c7f18b1ef6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:21.006208 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a69551-1082-4a28-a681-98c7f18b1ef6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:21.007649 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a69551-1082-4a28-a681-98c7f18b1ef6 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:21.018245 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1139/2255] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:56:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:56:21.049489 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187ebab-1eed-4eca-9d43-e694d3c2eacb None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:21.050937 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-8187ebab-1eed-4eca-9d43-e694d3c2eacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:21.063270 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187ebab-1eed-4eca-9d43-e694d3c2eacb None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:21.074802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b898f-b16a-4bf5-a5f5-4498eeeada6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:21.075627 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b898f-b16a-4bf5-a5f5-4498eeeada6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:21.076015 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b898f-b16a-4bf5-a5f5-4498eeeada6e admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:21.076392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b898f-b16a-4bf5-a5f5-4498eeeada6e admin admin] PATH_INFO: `/v3/projects/40a2b56f30e44eeeb3dcb11dced91a91` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:21.077734 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b898f-b16a-4bf5-a5f5-4498eeeada6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=40a2b56f30e44eeeb3dcb11dced91a91)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:21.080464 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b898f-b16a-4bf5-a5f5-4498eeeada6e admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:21.213574 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-a12b898f-b16a-4bf5-a5f5-4498eeeada6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40a2b56f30e44eeeb3dcb11dced91a91', 'request_id': 'req-a12b898f-b16a-4bf5-a5f5-4498eeeada6e'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:21.320212 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-a12b898f-b16a-4bf5-a5f5-4498eeeada6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40a2b56f30e44eeeb3dcb11dced91a91', 'request_id': 'req-a12b898f-b16a-4bf5-a5f5-4498eeeada6e'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:21.335959 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-a12b898f-b16a-4bf5-a5f5-4498eeeada6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40a2b56f30e44eeeb3dcb11dced91a91', 'request_id': 'req-a12b898f-b16a-4bf5-a5f5-4498eeeada6e'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:21.347194 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-a12b898f-b16a-4bf5-a5f5-4498eeeada6e admin admin] The token cache is being invalidate because project 40a2b56f30e44eeeb3dcb11dced91a91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:56:21.347488 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-a12b898f-b16a-4bf5-a5f5-4498eeeada6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:21.348166 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-a12b898f-b16a-4bf5-a5f5-4498eeeada6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:21.349986 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1117/2256] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:56:21 2022] DELETE /identity/v3/projects/40a2b56f30e44eeeb3dcb11dced91a91 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:56:21.354062 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a69551-1082-4a28-a681-98c7f18b1ef6 service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:21.366432 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a69551-1082-4a28-a681-98c7f18b1ef6 service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:21.419569 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:21.425941 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e79aa6-f863-4c15-9ed5-a9aba43ebb8e None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:21.430025 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e79aa6-f863-4c15-9ed5-a9aba43ebb8e None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:21.430025 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e79aa6-f863-4c15-9ed5-a9aba43ebb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:21.455335 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-07e79aa6-f863-4c15-9ed5-a9aba43ebb8e None None] MFA Rules not processed for user `cd45f1813efc4926a30454aaa108ac47`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:21.456690 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-07e79aa6-f863-4c15-9ed5-a9aba43ebb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:21.491026 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c549fd88-174d-4eec-aaa5-5914b7786c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:21.491973 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549fd88-174d-4eec-aaa5-5914b7786c22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:21.492512 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549fd88-174d-4eec-aaa5-5914b7786c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:21.492952 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549fd88-174d-4eec-aaa5-5914b7786c22 admin admin] PATH_INFO: `/v3/users/0ee92728522e4f60a860805da8e2cdc3` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:21.494380 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c549fd88-174d-4eec-aaa5-5914b7786c22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ee92728522e4f60a860805da8e2cdc3)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:21.497583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c549fd88-174d-4eec-aaa5-5914b7786c22 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:21.515249 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:21.535578 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1118/2257] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:56:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:56:21.536664 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e79aa6-f863-4c15-9ed5-a9aba43ebb8e None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:21.540679 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-07e79aa6-f863-4c15-9ed5-a9aba43ebb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:21.561015 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e79aa6-f863-4c15-9ed5-a9aba43ebb8e None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:21.568529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d000b8cf-a117-4dd0-a8b3-c71117f13db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:21.568529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000b8cf-a117-4dd0-a8b3-c71117f13db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:21.568529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000b8cf-a117-4dd0-a8b3-c71117f13db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:21.568529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000b8cf-a117-4dd0-a8b3-c71117f13db0 admin admin] PATH_INFO: `/v3/projects/29601c525a8d42878760f5ae2c4eb580` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:21.569036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d000b8cf-a117-4dd0-a8b3-c71117f13db0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29601c525a8d42878760f5ae2c4eb580)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:21.573811 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d000b8cf-a117-4dd0-a8b3-c71117f13db0 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:21.936552 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-d000b8cf-a117-4dd0-a8b3-c71117f13db0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29601c525a8d42878760f5ae2c4eb580', 'request_id': 'req-d000b8cf-a117-4dd0-a8b3-c71117f13db0'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:21.936773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-c549fd88-174d-4eec-aaa5-5914b7786c22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ee92728522e4f60a860805da8e2cdc3'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:21.937463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-c549fd88-174d-4eec-aaa5-5914b7786c22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ee92728522e4f60a860805da8e2cdc3'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:21.950391 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-c549fd88-174d-4eec-aaa5-5914b7786c22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ee92728522e4f60a860805da8e2cdc3'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:22.000855 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-d000b8cf-a117-4dd0-a8b3-c71117f13db0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29601c525a8d42878760f5ae2c4eb580', 'request_id': 'req-d000b8cf-a117-4dd0-a8b3-c71117f13db0'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:22.007588 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-d000b8cf-a117-4dd0-a8b3-c71117f13db0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29601c525a8d42878760f5ae2c4eb580', 'request_id': 'req-d000b8cf-a117-4dd0-a8b3-c71117f13db0'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:22.018088 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-d000b8cf-a117-4dd0-a8b3-c71117f13db0 admin admin] The token cache is being invalidate because project 29601c525a8d42878760f5ae2c4eb580 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:56:22.018088 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-d000b8cf-a117-4dd0-a8b3-c71117f13db0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:22.018755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-d000b8cf-a117-4dd0-a8b3-c71117f13db0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:22.020235 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1119/2258] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:56:21 2022] DELETE /identity/v3/projects/29601c525a8d42878760f5ae2c4eb580 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:56:22.024875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9981e35-2af8-4247-84db-0c99b052e434 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:22.025153 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9981e35-2af8-4247-84db-0c99b052e434 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:22.025429 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9981e35-2af8-4247-84db-0c99b052e434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:22.046898 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-e9981e35-2af8-4247-84db-0c99b052e434 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:22.051944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-e9981e35-2af8-4247-84db-0c99b052e434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:22.069190 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-c549fd88-174d-4eec-aaa5-5914b7786c22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ee92728522e4f60a860805da8e2cdc3'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:22.069835 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-c549fd88-174d-4eec-aaa5-5914b7786c22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ee92728522e4f60a860805da8e2cdc3'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:22.095960 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1140/2259] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:56:21 2022] DELETE /identity/v3/users/0ee92728522e4f60a860805da8e2cdc3 => generated 0 bytes in 733 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:56:22.139806 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1120/2260] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:56:22.149468 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5e787-01de-44d8-850c-835e265805a2 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:22.149468 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5e787-01de-44d8-850c-835e265805a2 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:22.149809 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5e787-01de-44d8-850c-835e265805a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:22.178458 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-baf5e787-01de-44d8-850c-835e265805a2 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:22.178458 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-baf5e787-01de-44d8-850c-835e265805a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:22.247058 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1141/2261] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:56:22.252694 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9981e35-2af8-4247-84db-0c99b052e434 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:22.261255 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9981e35-2af8-4247-84db-0c99b052e434 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:22.265806 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7419e4-dc50-4b4e-812b-887acfe1195f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:22.266402 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7419e4-dc50-4b4e-812b-887acfe1195f admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:22.266683 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7419e4-dc50-4b4e-812b-887acfe1195f admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:22.267546 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7419e4-dc50-4b4e-812b-887acfe1195f admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:22.268286 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7419e4-dc50-4b4e-812b-887acfe1195f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:56:22.268567 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7419e4-dc50-4b4e-812b-887acfe1195f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:22.270162 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7419e4-dc50-4b4e-812b-887acfe1195f admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:22.281440 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1121/2262] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 00:56:22.286433 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5e787-01de-44d8-850c-835e265805a2 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:22.295925 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5e787-01de-44d8-850c-835e265805a2 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:22.300801 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db58a76-ded1-4cfe-a0a6-cc9df3df67d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:22.301986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db58a76-ded1-4cfe-a0a6-cc9df3df67d2 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:22.301986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db58a76-ded1-4cfe-a0a6-cc9df3df67d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:22.301986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db58a76-ded1-4cfe-a0a6-cc9df3df67d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:22.302575 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db58a76-ded1-4cfe-a0a6-cc9df3df67d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:22.304540 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db58a76-ded1-4cfe-a0a6-cc9df3df67d2 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:22.401840 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1142/2263] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:56:22 2022] POST /identity/v3/projects => generated 363 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:56:22.402927 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7419e4-dc50-4b4e-812b-887acfe1195f admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:22.422574 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7419e4-dc50-4b4e-812b-887acfe1195f admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:22.525349 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9e6bf-43c5-4d32-a66f-16c88cb38947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:22.526120 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9e6bf-43c5-4d32-a66f-16c88cb38947 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:22.526487 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9e6bf-43c5-4d32-a66f-16c88cb38947 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:22.526878 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9e6bf-43c5-4d32-a66f-16c88cb38947 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:22.528027 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9e6bf-43c5-4d32-a66f-16c88cb38947 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:22.530849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9e6bf-43c5-4d32-a66f-16c88cb38947 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:22.543886 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:22.660616 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db58a76-ded1-4cfe-a0a6-cc9df3df67d2 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:22.663383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-3db58a76-ded1-4cfe-a0a6-cc9df3df67d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:22.677773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db58a76-ded1-4cfe-a0a6-cc9df3df67d2 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:22.682852 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a34b44-f7f2-447b-8e23-20f31ab702e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:22.683492 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a34b44-f7f2-447b-8e23-20f31ab702e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:22.683775 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a34b44-f7f2-447b-8e23-20f31ab702e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:22.684057 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a34b44-f7f2-447b-8e23-20f31ab702e5 admin admin] PATH_INFO: `/v3/projects/2fdb9bff842f421baa9660fde546c405` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:22.685236 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a34b44-f7f2-447b-8e23-20f31ab702e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fdb9bff842f421baa9660fde546c405)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:22.687397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a34b44-f7f2-447b-8e23-20f31ab702e5 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:22.713705 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:22.728246 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1122/2264] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:56:22 2022] POST /identity/v3/users => generated 282 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:56:22.732166 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9e6bf-43c5-4d32-a66f-16c88cb38947 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:22.742762 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9e6bf-43c5-4d32-a66f-16c88cb38947 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:22.749044 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04100b8f-33cc-4969-973d-a2d21623862b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:22.749819 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04100b8f-33cc-4969-973d-a2d21623862b admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:22.750105 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04100b8f-33cc-4969-973d-a2d21623862b admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:22.750379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04100b8f-33cc-4969-973d-a2d21623862b admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:22.751028 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04100b8f-33cc-4969-973d-a2d21623862b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:22.752681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04100b8f-33cc-4969-973d-a2d21623862b admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:22.765014 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1123/2265] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 00:56:22.768784 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04100b8f-33cc-4969-973d-a2d21623862b admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:22.784776 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04100b8f-33cc-4969-973d-a2d21623862b admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:22.795032 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c517ed4e-94ae-4af7-8d5c-37f39364b0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:22.795994 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c517ed4e-94ae-4af7-8d5c-37f39364b0bc admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:22.796272 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c517ed4e-94ae-4af7-8d5c-37f39364b0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:22.796566 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c517ed4e-94ae-4af7-8d5c-37f39364b0bc admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:22.797442 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c517ed4e-94ae-4af7-8d5c-37f39364b0bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:22.799732 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c517ed4e-94ae-4af7-8d5c-37f39364b0bc admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:22.852074 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-62a34b44-f7f2-447b-8e23-20f31ab702e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fdb9bff842f421baa9660fde546c405', 'request_id': 'req-62a34b44-f7f2-447b-8e23-20f31ab702e5'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:22.854337 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1124/2266] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 00:56:22.864288 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c517ed4e-94ae-4af7-8d5c-37f39364b0bc admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:22.871606 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c517ed4e-94ae-4af7-8d5c-37f39364b0bc admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:22.938480 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-62a34b44-f7f2-447b-8e23-20f31ab702e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fdb9bff842f421baa9660fde546c405', 'request_id': 'req-62a34b44-f7f2-447b-8e23-20f31ab702e5'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:22.947401 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-62a34b44-f7f2-447b-8e23-20f31ab702e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fdb9bff842f421baa9660fde546c405', 'request_id': 'req-62a34b44-f7f2-447b-8e23-20f31ab702e5'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:22.977292 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-62a34b44-f7f2-447b-8e23-20f31ab702e5 admin admin] The token cache is being invalidate because project 2fdb9bff842f421baa9660fde546c405 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:56:22.978075 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-62a34b44-f7f2-447b-8e23-20f31ab702e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:22.978910 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-62a34b44-f7f2-447b-8e23-20f31ab702e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:22.984866 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1143/2267] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:56:22 2022] DELETE /identity/v3/projects/2fdb9bff842f421baa9660fde546c405 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:56:22.987351 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe663f37-8722-4ee1-bd57-c967202fd4e5 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:22.987744 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe663f37-8722-4ee1-bd57-c967202fd4e5 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:22.988464 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe663f37-8722-4ee1-bd57-c967202fd4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:22.999850 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4aea4a-6310-48c6-835d-a6821e2f1a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:23.001278 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4aea4a-6310-48c6-835d-a6821e2f1a59 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:23.003256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4aea4a-6310-48c6-835d-a6821e2f1a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:23.003707 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4aea4a-6310-48c6-835d-a6821e2f1a59 admin admin] PATH_INFO: `/v3/projects/457bcaaea73f487e97bb76fc3e1f6870/users/92c33a46fce541de835acb37b0b4057d/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:23.013897 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-fe663f37-8722-4ee1-bd57-c967202fd4e5 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:23.017813 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-fe663f37-8722-4ee1-bd57-c967202fd4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:23.036723 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4aea4a-6310-48c6-835d-a6821e2f1a59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=457bcaaea73f487e97bb76fc3e1f6870, user_id=92c33a46fce541de835acb37b0b4057d, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:23.036723 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4aea4a-6310-48c6-835d-a6821e2f1a59 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:23.052578 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1144/2268] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:56:23.057486 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a62fd6-301e-4394-a37f-1b56a984b09f None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:23.058048 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a62fd6-301e-4394-a37f-1b56a984b09f None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:23.058172 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a62fd6-301e-4394-a37f-1b56a984b09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:23.082989 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-52a62fd6-301e-4394-a37f-1b56a984b09f None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:23.084186 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-52a62fd6-301e-4394-a37f-1b56a984b09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:23.095550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1145/2269] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:56:23.096492 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a62fd6-301e-4394-a37f-1b56a984b09f None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:23.106549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a62fd6-301e-4394-a37f-1b56a984b09f None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:23.118101 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1125/2270] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:56:22 2022] PUT /identity/v3/projects/457bcaaea73f487e97bb76fc3e1f6870/users/92c33a46fce541de835acb37b0b4057d/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:56:23.129071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d8dc8-9d24-4821-a766-12aa807d9040 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:23.129368 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d8dc8-9d24-4821-a766-12aa807d9040 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:23.129675 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d8dc8-9d24-4821-a766-12aa807d9040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:23.165078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-193d8dc8-9d24-4821-a766-12aa807d9040 None None] MFA Rules not processed for user `92c33a46fce541de835acb37b0b4057d`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:23.166646 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-193d8dc8-9d24-4821-a766-12aa807d9040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:23.175549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00752c13-00ff-4d08-b174-47f307f6dd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:23.176209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00752c13-00ff-4d08-b174-47f307f6dd61 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:23.176513 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00752c13-00ff-4d08-b174-47f307f6dd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:23.176916 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00752c13-00ff-4d08-b174-47f307f6dd61 admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:23.177636 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00752c13-00ff-4d08-b174-47f307f6dd61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:56:23.177950 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00752c13-00ff-4d08-b174-47f307f6dd61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:23.179568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00752c13-00ff-4d08-b174-47f307f6dd61 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:23.194361 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1146/2271] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:56:23.198278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00752c13-00ff-4d08-b174-47f307f6dd61 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:23.214052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00752c13-00ff-4d08-b174-47f307f6dd61 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:23.221753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b290df6-d42b-4a33-9e7a-6e4d9ec224a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:23.222390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b290df6-d42b-4a33-9e7a-6e4d9ec224a6 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:23.223610 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b290df6-d42b-4a33-9e7a-6e4d9ec224a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:23.223767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b290df6-d42b-4a33-9e7a-6e4d9ec224a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:23.229253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b290df6-d42b-4a33-9e7a-6e4d9ec224a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:23.229253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b290df6-d42b-4a33-9e7a-6e4d9ec224a6 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:23.254497 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1126/2272] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:56:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:56:23.269107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d8dc8-9d24-4821-a766-12aa807d9040 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:23.270702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-193d8dc8-9d24-4821-a766-12aa807d9040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:23.327795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1147/2273] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:56:23 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:56:23.328933 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b290df6-d42b-4a33-9e7a-6e4d9ec224a6 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:23.339762 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d8dc8-9d24-4821-a766-12aa807d9040 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:23.396491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c37c46-f601-4bfe-8772-28e8684657cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:23.401452 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c37c46-f601-4bfe-8772-28e8684657cd service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:23.401452 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c37c46-f601-4bfe-8772-28e8684657cd service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:23.401452 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c37c46-f601-4bfe-8772-28e8684657cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:23.404741 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b290df6-d42b-4a33-9e7a-6e4d9ec224a6 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:23.420252 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba26fcc-bf49-45ab-bc41-e342d41326d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:23.421181 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba26fcc-bf49-45ab-bc41-e342d41326d5 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:23.421662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba26fcc-bf49-45ab-bc41-e342d41326d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:23.422066 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba26fcc-bf49-45ab-bc41-e342d41326d5 admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:23.422787 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba26fcc-bf49-45ab-bc41-e342d41326d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:23.424257 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:23.425496 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba26fcc-bf49-45ab-bc41-e342d41326d5 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:23.474189 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c37c46-f601-4bfe-8772-28e8684657cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:23.476171 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c37c46-f601-4bfe-8772-28e8684657cd service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:23.495693 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1127/2274] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:56:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:56:23.497411 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721cdb4-687e-4836-a2aa-05433d73e1dd None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:23.497896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721cdb4-687e-4836-a2aa-05433d73e1dd None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:23.498383 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721cdb4-687e-4836-a2aa-05433d73e1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:23.520848 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-2721cdb4-687e-4836-a2aa-05433d73e1dd None None] MFA Rules not processed for user `bbd9bbb083ab4f5e84e135ec5103cdae`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:23.522411 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-2721cdb4-687e-4836-a2aa-05433d73e1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:23.558501 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1148/2275] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:56:23 2022] POST /identity/v3/users => generated 292 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:56:23.561346 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba26fcc-bf49-45ab-bc41-e342d41326d5 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:23.582422 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba26fcc-bf49-45ab-bc41-e342d41326d5 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:23.605842 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1128/2276] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Aug 26 00:56:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:56:23.614081 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:23.637305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c8e11c-4fe8-4ec4-986e-3eed11175be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:23.638561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8e11c-4fe8-4ec4-986e-3eed11175be9 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:23.638987 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8e11c-4fe8-4ec4-986e-3eed11175be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:23.639415 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8e11c-4fe8-4ec4-986e-3eed11175be9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:23.640357 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c8e11c-4fe8-4ec4-986e-3eed11175be9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:23.642301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c8e11c-4fe8-4ec4-986e-3eed11175be9 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:23.658937 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1149/2277] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:56:23.663135 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721cdb4-687e-4836-a2aa-05433d73e1dd None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:23.670416 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c8e11c-4fe8-4ec4-986e-3eed11175be9 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:23.673828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-77c8e11c-4fe8-4ec4-986e-3eed11175be9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:23.675901 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721cdb4-687e-4836-a2aa-05433d73e1dd None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:23.681576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b46df-fc85-4804-9144-08e36b888f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:23.682399 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b46df-fc85-4804-9144-08e36b888f59 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:23.682681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b46df-fc85-4804-9144-08e36b888f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:23.682963 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b46df-fc85-4804-9144-08e36b888f59 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:23.684024 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b46df-fc85-4804-9144-08e36b888f59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:23.685263 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b46df-fc85-4804-9144-08e36b888f59 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:23.697070 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1129/2278] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:56:23.701376 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b46df-fc85-4804-9144-08e36b888f59 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:23.711728 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b46df-fc85-4804-9144-08e36b888f59 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:23.717368 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba121f4-60d2-4fe1-8e1d-fcf0ecdc14f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:23.718035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba121f4-60d2-4fe1-8e1d-fcf0ecdc14f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:23.718319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba121f4-60d2-4fe1-8e1d-fcf0ecdc14f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:23.718597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba121f4-60d2-4fe1-8e1d-fcf0ecdc14f1 admin admin] PATH_INFO: `/v3/projects/ea8a7d2bf6b94d769101bdaed41a4d75/users/32bac73d59f941c48c9e0e0460b10823/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:23.736732 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba121f4-60d2-4fe1-8e1d-fcf0ecdc14f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea8a7d2bf6b94d769101bdaed41a4d75, user_id=32bac73d59f941c48c9e0e0460b10823, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:23.739917 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba121f4-60d2-4fe1-8e1d-fcf0ecdc14f1 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:23.759936 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c8e11c-4fe8-4ec4-986e-3eed11175be9 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:23.765313 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f6914-eaef-4f7c-b700-2a7c0e459bc3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f700502671446c797a566f07398a9fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:23.765965 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f6914-eaef-4f7c-b700-2a7c0e459bc3 service swift] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:23.766243 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f6914-eaef-4f7c-b700-2a7c0e459bc3 service swift] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:23.766514 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f6914-eaef-4f7c-b700-2a7c0e459bc3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:23.782378 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f6914-eaef-4f7c-b700-2a7c0e459bc3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:23.782378 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f6914-eaef-4f7c-b700-2a7c0e459bc3 service swift] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:23.786207 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1150/2279] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Aug 26 00:56:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 00:56:23.866908 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1130/2280] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:56:23 2022] PUT /identity/v3/projects/ea8a7d2bf6b94d769101bdaed41a4d75/users/32bac73d59f941c48c9e0e0460b10823/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:56:23.882406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da1578-4cd0-44e1-b1ff-31f50e74f825 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:23.882692 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da1578-4cd0-44e1-b1ff-31f50e74f825 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:23.882972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da1578-4cd0-44e1-b1ff-31f50e74f825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:23.923233 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-22da1578-4cd0-44e1-b1ff-31f50e74f825 None None] MFA Rules not processed for user `32bac73d59f941c48c9e0e0460b10823`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:23.924716 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-22da1578-4cd0-44e1-b1ff-31f50e74f825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:24.008013 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1151/2281] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:56:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:56:24.020186 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba121f4-60d2-4fe1-8e1d-fcf0ecdc14f1 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:24.025168 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-9ba121f4-60d2-4fe1-8e1d-fcf0ecdc14f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:24.139067 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba121f4-60d2-4fe1-8e1d-fcf0ecdc14f1 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:24.143995 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd53dd-7327-490b-903f-102d491b6bf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:24.144838 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd53dd-7327-490b-903f-102d491b6bf6 service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:24.145212 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd53dd-7327-490b-903f-102d491b6bf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:24.145598 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd53dd-7327-490b-903f-102d491b6bf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:24.154898 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd53dd-7327-490b-903f-102d491b6bf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:24.156346 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd53dd-7327-490b-903f-102d491b6bf6 service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:24.170332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1131/2282] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:56:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:56:24.424180 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:28.664955 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a40387-6a03-4dc5-997a-7d60b430eeea None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:28.665872 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a40387-6a03-4dc5-997a-7d60b430eeea None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:28.665872 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a40387-6a03-4dc5-997a-7d60b430eeea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:28.692612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-76a40387-6a03-4dc5-997a-7d60b430eeea None None] MFA Rules not processed for user `0cc36def73aa4bedb9e79d4cf11f5e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:28.694297 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-76a40387-6a03-4dc5-997a-7d60b430eeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:28.811756 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1152/2283] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:56:28 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:56:34.829323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735911a-b41a-4c06-8e12-f46233c40efe None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:34.832885 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735911a-b41a-4c06-8e12-f46233c40efe None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:34.832885 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735911a-b41a-4c06-8e12-f46233c40efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:34.858326 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-f735911a-b41a-4c06-8e12-f46233c40efe None None] MFA Rules not processed for user `bbd9bbb083ab4f5e84e135ec5103cdae`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:34.858326 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-f735911a-b41a-4c06-8e12-f46233c40efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:34.979866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1132/2284] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Aug 26 00:56:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:56:35.041113 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a40387-6a03-4dc5-997a-7d60b430eeea None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:35.065522 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:35.144614 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a40387-6a03-4dc5-997a-7d60b430eeea None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:35.150931 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef3b69-4edf-491d-9394-b6698fa666b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f700502671446c797a566f07398a9fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:35.156223 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef3b69-4edf-491d-9394-b6698fa666b3 service swift] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:35.156223 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef3b69-4edf-491d-9394-b6698fa666b3 service swift] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:35.156223 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef3b69-4edf-491d-9394-b6698fa666b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:35.174298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef3b69-4edf-491d-9394-b6698fa666b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:35.176973 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef3b69-4edf-491d-9394-b6698fa666b3 service swift] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:35.211048 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1153/2285] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Aug 26 00:56:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 00:56:39.572337 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735911a-b41a-4c06-8e12-f46233c40efe None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:39.708395 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735911a-b41a-4c06-8e12-f46233c40efe None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:39.716628 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140435b2-d29a-48f2-ac3e-7fa42f9f53ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:39.716628 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140435b2-d29a-48f2-ac3e-7fa42f9f53ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:39.716628 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140435b2-d29a-48f2-ac3e-7fa42f9f53ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:39.716628 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140435b2-d29a-48f2-ac3e-7fa42f9f53ca admin admin] PATH_INFO: `/v3/users/32bac73d59f941c48c9e0e0460b10823` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:39.717106 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140435b2-d29a-48f2-ac3e-7fa42f9f53ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=32bac73d59f941c48c9e0e0460b10823)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:39.718046 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140435b2-d29a-48f2-ac3e-7fa42f9f53ca admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:39.998694 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-140435b2-d29a-48f2-ac3e-7fa42f9f53ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32bac73d59f941c48c9e0e0460b10823'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:40.012383 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-140435b2-d29a-48f2-ac3e-7fa42f9f53ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32bac73d59f941c48c9e0e0460b10823'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:40.081700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-140435b2-d29a-48f2-ac3e-7fa42f9f53ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32bac73d59f941c48c9e0e0460b10823'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:40.082430 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-140435b2-d29a-48f2-ac3e-7fa42f9f53ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32bac73d59f941c48c9e0e0460b10823'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:40.083134 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-140435b2-d29a-48f2-ac3e-7fa42f9f53ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32bac73d59f941c48c9e0e0460b10823'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:40.089210 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:40.106406 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1133/2286] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:56:39 2022] DELETE /identity/v3/users/32bac73d59f941c48c9e0e0460b10823 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:56:40.116413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f4a5f-bdff-4774-9867-8d005a284259 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:40.116413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f4a5f-bdff-4774-9867-8d005a284259 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:40.116413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f4a5f-bdff-4774-9867-8d005a284259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:40.144795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-6e7f4a5f-bdff-4774-9867-8d005a284259 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:40.146317 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-6e7f4a5f-bdff-4774-9867-8d005a284259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:40.239520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1154/2287] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:56:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:56:40.249327 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140435b2-d29a-48f2-ac3e-7fa42f9f53ca admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:40.251468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-140435b2-d29a-48f2-ac3e-7fa42f9f53ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:40.397155 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140435b2-d29a-48f2-ac3e-7fa42f9f53ca admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:40.400960 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d133a84-b0d8-4b2c-b223-318097d60eab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:40.401529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d133a84-b0d8-4b2c-b223-318097d60eab service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:40.401810 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d133a84-b0d8-4b2c-b223-318097d60eab service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:40.402084 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d133a84-b0d8-4b2c-b223-318097d60eab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:40.413652 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d133a84-b0d8-4b2c-b223-318097d60eab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:40.415679 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d133a84-b0d8-4b2c-b223-318097d60eab service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:40.441832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1134/2288] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:56:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:56:41.192462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f4a5f-bdff-4774-9867-8d005a284259 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:41.196181 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-6e7f4a5f-bdff-4774-9867-8d005a284259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:41.245187 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f4a5f-bdff-4774-9867-8d005a284259 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:41.260627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb6142-09ba-4766-bb3d-d1bdf9f75df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:41.261591 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb6142-09ba-4766-bb3d-d1bdf9f75df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:41.262118 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb6142-09ba-4766-bb3d-d1bdf9f75df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:41.262573 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb6142-09ba-4766-bb3d-d1bdf9f75df3 admin admin] PATH_INFO: `/v3/projects/ea8a7d2bf6b94d769101bdaed41a4d75` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:41.264347 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb6142-09ba-4766-bb3d-d1bdf9f75df3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea8a7d2bf6b94d769101bdaed41a4d75)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:41.270816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb6142-09ba-4766-bb3d-d1bdf9f75df3 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:41.410874 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-fafb6142-09ba-4766-bb3d-d1bdf9f75df3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea8a7d2bf6b94d769101bdaed41a4d75', 'request_id': 'req-fafb6142-09ba-4766-bb3d-d1bdf9f75df3'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:41.488412 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-fafb6142-09ba-4766-bb3d-d1bdf9f75df3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea8a7d2bf6b94d769101bdaed41a4d75', 'request_id': 'req-fafb6142-09ba-4766-bb3d-d1bdf9f75df3'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:41.497334 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-fafb6142-09ba-4766-bb3d-d1bdf9f75df3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea8a7d2bf6b94d769101bdaed41a4d75', 'request_id': 'req-fafb6142-09ba-4766-bb3d-d1bdf9f75df3'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:41.520297 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-fafb6142-09ba-4766-bb3d-d1bdf9f75df3 admin admin] The token cache is being invalidate because project ea8a7d2bf6b94d769101bdaed41a4d75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:56:41.530179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-fafb6142-09ba-4766-bb3d-d1bdf9f75df3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:41.530179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-fafb6142-09ba-4766-bb3d-d1bdf9f75df3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:41.530179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1155/2289] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:56:41 2022] DELETE /identity/v3/projects/ea8a7d2bf6b94d769101bdaed41a4d75 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:56:41.530725 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8cc2c-3cdc-4517-8ee7-39f71db756ae None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:41.530725 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8cc2c-3cdc-4517-8ee7-39f71db756ae None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:41.530725 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8cc2c-3cdc-4517-8ee7-39f71db756ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:41.557475 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-13a8cc2c-3cdc-4517-8ee7-39f71db756ae None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:41.558786 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-13a8cc2c-3cdc-4517-8ee7-39f71db756ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:41.643524 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1135/2290] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:56:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:56:41.650085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c19a1f-a182-4e74-b962-58a05130a20f None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:41.650376 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c19a1f-a182-4e74-b962-58a05130a20f None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:41.650655 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c19a1f-a182-4e74-b962-58a05130a20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:41.693272 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-a5c19a1f-a182-4e74-b962-58a05130a20f None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:41.694650 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a5c19a1f-a182-4e74-b962-58a05130a20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:41.714697 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1156/2291] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:56:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:56:41.721418 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8cc2c-3cdc-4517-8ee7-39f71db756ae None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:41.738604 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8cc2c-3cdc-4517-8ee7-39f71db756ae None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:41.755837 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674a99d-75e5-4ed3-a132-f471cfcf0f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:41.755837 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674a99d-75e5-4ed3-a132-f471cfcf0f26 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:41.756312 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674a99d-75e5-4ed3-a132-f471cfcf0f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:41.756716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674a99d-75e5-4ed3-a132-f471cfcf0f26 admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:41.757506 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8674a99d-75e5-4ed3-a132-f471cfcf0f26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:56:41.757939 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8674a99d-75e5-4ed3-a132-f471cfcf0f26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:41.759733 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8674a99d-75e5-4ed3-a132-f471cfcf0f26 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:41.786785 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1136/2292] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:56:41.791830 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c19a1f-a182-4e74-b962-58a05130a20f None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:41.807449 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c19a1f-a182-4e74-b962-58a05130a20f None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:41.820464 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20455a6e-978f-44ae-9039-b28c6d6198b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:41.821190 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20455a6e-978f-44ae-9039-b28c6d6198b4 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:41.821493 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20455a6e-978f-44ae-9039-b28c6d6198b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:41.821798 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20455a6e-978f-44ae-9039-b28c6d6198b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:41.822478 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20455a6e-978f-44ae-9039-b28c6d6198b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:41.824462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20455a6e-978f-44ae-9039-b28c6d6198b4 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:41.939070 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:41.952335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1157/2293] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:56:41 2022] POST /identity/v3/projects => generated 377 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:56:41.958557 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674a99d-75e5-4ed3-a132-f471cfcf0f26 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:41.974999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674a99d-75e5-4ed3-a132-f471cfcf0f26 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:42.046903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a915a88-68fc-462f-8ecb-573f5334d9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:42.047462 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a915a88-68fc-462f-8ecb-573f5334d9c8 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:42.048408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a915a88-68fc-462f-8ecb-573f5334d9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:42.048630 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a915a88-68fc-462f-8ecb-573f5334d9c8 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:42.049269 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a915a88-68fc-462f-8ecb-573f5334d9c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:42.051325 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a915a88-68fc-462f-8ecb-573f5334d9c8 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:42.160923 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1137/2294] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:56:41 2022] POST /identity/v3/users => generated 289 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:56:42.169158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20455a6e-978f-44ae-9039-b28c6d6198b4 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:42.176671 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20455a6e-978f-44ae-9039-b28c6d6198b4 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:42.200508 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036346c-eca9-495f-a719-19a8c23d1d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:42.200508 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036346c-eca9-495f-a719-19a8c23d1d8d admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:42.200508 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036346c-eca9-495f-a719-19a8c23d1d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:42.200508 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036346c-eca9-495f-a719-19a8c23d1d8d admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:42.201572 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036346c-eca9-495f-a719-19a8c23d1d8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:42.201572 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036346c-eca9-495f-a719-19a8c23d1d8d admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:42.218566 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1158/2295] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 00:56:42.220730 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a915a88-68fc-462f-8ecb-573f5334d9c8 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:42.233778 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a915a88-68fc-462f-8ecb-573f5334d9c8 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:42.238627 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f33a98-f575-4ce0-a53e-fc6a724c750b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:42.239348 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f33a98-f575-4ce0-a53e-fc6a724c750b admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:42.239720 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f33a98-f575-4ce0-a53e-fc6a724c750b admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:42.240101 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f33a98-f575-4ce0-a53e-fc6a724c750b admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:42.242394 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f33a98-f575-4ce0-a53e-fc6a724c750b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:42.244108 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f33a98-f575-4ce0-a53e-fc6a724c750b admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:42.259817 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1138/2296] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:56:42.263678 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036346c-eca9-495f-a719-19a8c23d1d8d admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:42.280884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036346c-eca9-495f-a719-19a8c23d1d8d admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:42.287244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc58fad-116b-4921-b6d3-18ef52f49e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:42.288366 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc58fad-116b-4921-b6d3-18ef52f49e52 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:42.288724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc58fad-116b-4921-b6d3-18ef52f49e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:42.289090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc58fad-116b-4921-b6d3-18ef52f49e52 admin admin] PATH_INFO: `/v3/projects/8b67076aed8a4fd59717b7706b1be4b4/users/cc6c20be27a14a08808e10fc843afa82/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:42.306475 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc58fad-116b-4921-b6d3-18ef52f49e52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b67076aed8a4fd59717b7706b1be4b4, user_id=cc6c20be27a14a08808e10fc843afa82, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:42.309714 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc58fad-116b-4921-b6d3-18ef52f49e52 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:42.402982 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1159/2297] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:56:42 2022] PUT /identity/v3/projects/8b67076aed8a4fd59717b7706b1be4b4/users/cc6c20be27a14a08808e10fc843afa82/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:56:42.414803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a323ea-84ec-4e38-9f63-1123de657be1 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:42.415195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a323ea-84ec-4e38-9f63-1123de657be1 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:42.415498 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a323ea-84ec-4e38-9f63-1123de657be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:42.424291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:42.460574 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-35a323ea-84ec-4e38-9f63-1123de657be1 None None] MFA Rules not processed for user `cc6c20be27a14a08808e10fc843afa82`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:42.462931 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-35a323ea-84ec-4e38-9f63-1123de657be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:42.537225 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1139/2298] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:56:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:56:42.542472 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc58fad-116b-4921-b6d3-18ef52f49e52 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:42.621014 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc58fad-116b-4921-b6d3-18ef52f49e52 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:42.626451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb07bb3-1ffe-4e94-b61b-db5b97f6f312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:42.627182 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb07bb3-1ffe-4e94-b61b-db5b97f6f312 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:42.627785 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb07bb3-1ffe-4e94-b61b-db5b97f6f312 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:42.628166 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb07bb3-1ffe-4e94-b61b-db5b97f6f312 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:42.628842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb07bb3-1ffe-4e94-b61b-db5b97f6f312 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:42.630819 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb07bb3-1ffe-4e94-b61b-db5b97f6f312 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:42.753664 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1160/2299] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:56:42 2022] POST /identity/v3/projects => generated 377 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:56:42.758733 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a323ea-84ec-4e38-9f63-1123de657be1 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:42.781893 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a323ea-84ec-4e38-9f63-1123de657be1 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:42.857242 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479f22b-06b6-4b8b-bf17-e873831cdc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:42.857930 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479f22b-06b6-4b8b-bf17-e873831cdc13 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:42.858426 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479f22b-06b6-4b8b-bf17-e873831cdc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:42.858589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479f22b-06b6-4b8b-bf17-e873831cdc13 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:42.859327 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e479f22b-06b6-4b8b-bf17-e873831cdc13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:42.861676 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e479f22b-06b6-4b8b-bf17-e873831cdc13 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:42.986714 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1140/2300] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:56:42 2022] POST /identity/v3/users => generated 289 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:56:42.987773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb07bb3-1ffe-4e94-b61b-db5b97f6f312 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:43.003129 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb07bb3-1ffe-4e94-b61b-db5b97f6f312 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:43.015240 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1fc4f9-b49d-4780-b3e6-3e7f91e894a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:43.016445 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1fc4f9-b49d-4780-b3e6-3e7f91e894a5 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:43.016931 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1fc4f9-b49d-4780-b3e6-3e7f91e894a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:43.021007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1fc4f9-b49d-4780-b3e6-3e7f91e894a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:43.021717 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1fc4f9-b49d-4780-b3e6-3e7f91e894a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:43.023353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1fc4f9-b49d-4780-b3e6-3e7f91e894a5 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:43.046501 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1161/2301] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:56:43.050122 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479f22b-06b6-4b8b-bf17-e873831cdc13 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:43.060978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479f22b-06b6-4b8b-bf17-e873831cdc13 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:43.067135 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ff5df-76e6-4066-b494-932cd2601a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:43.067560 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ff5df-76e6-4066-b494-932cd2601a35 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:43.067902 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ff5df-76e6-4066-b494-932cd2601a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:43.068184 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ff5df-76e6-4066-b494-932cd2601a35 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:43.068867 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ff5df-76e6-4066-b494-932cd2601a35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:43.070540 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ff5df-76e6-4066-b494-932cd2601a35 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:43.086240 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1141/2302] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:56:43.086993 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1fc4f9-b49d-4780-b3e6-3e7f91e894a5 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:43.101354 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1fc4f9-b49d-4780-b3e6-3e7f91e894a5 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:43.127914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd90100-e939-4dce-bf29-fbeb5196087c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:43.129075 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd90100-e939-4dce-bf29-fbeb5196087c admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:43.129576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd90100-e939-4dce-bf29-fbeb5196087c admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:43.129854 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd90100-e939-4dce-bf29-fbeb5196087c admin admin] PATH_INFO: `/v3/projects/254a371abdc34c90bcf25d7231e728f2/users/0e1b1621c6844d82ac73bfdc7887c5ee/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:43.150603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd90100-e939-4dce-bf29-fbeb5196087c admin admin] RBAC: Authorizing `identity:create_grant(project_id=254a371abdc34c90bcf25d7231e728f2, user_id=0e1b1621c6844d82ac73bfdc7887c5ee, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:43.154041 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd90100-e939-4dce-bf29-fbeb5196087c admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:43.287303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0352e65-f6f1-417b-9b7e-80036725e967 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:43.287841 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0352e65-f6f1-417b-9b7e-80036725e967 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:43.288121 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0352e65-f6f1-417b-9b7e-80036725e967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:43.289991 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1162/2303] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:56:43 2022] PUT /identity/v3/projects/254a371abdc34c90bcf25d7231e728f2/users/0e1b1621c6844d82ac73bfdc7887c5ee/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:56:43.331704 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-d0352e65-f6f1-417b-9b7e-80036725e967 None None] MFA Rules not processed for user `0e1b1621c6844d82ac73bfdc7887c5ee`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:43.333145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-d0352e65-f6f1-417b-9b7e-80036725e967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:43.428009 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:43.436539 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1142/2304] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:56:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:56:43.442696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd90100-e939-4dce-bf29-fbeb5196087c admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:43.532363 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd90100-e939-4dce-bf29-fbeb5196087c admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:43.548496 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66a5d5-47b7-4d80-bb80-fa8ac68ec415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:43.548496 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66a5d5-47b7-4d80-bb80-fa8ac68ec415 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:43.548496 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66a5d5-47b7-4d80-bb80-fa8ac68ec415 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:43.548496 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66a5d5-47b7-4d80-bb80-fa8ac68ec415 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:43.549035 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc66a5d5-47b7-4d80-bb80-fa8ac68ec415 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:43.549035 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc66a5d5-47b7-4d80-bb80-fa8ac68ec415 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:43.694933 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:43.708570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1163/2305] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:56:43 2022] POST /identity/v3/projects => generated 377 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:56:43.708762 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0352e65-f6f1-417b-9b7e-80036725e967 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:43.722817 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0352e65-f6f1-417b-9b7e-80036725e967 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:43.791897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300fd14-d68c-4858-8f5a-6d9d94b29bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:43.792559 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300fd14-d68c-4858-8f5a-6d9d94b29bd8 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:43.792851 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300fd14-d68c-4858-8f5a-6d9d94b29bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:43.793129 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300fd14-d68c-4858-8f5a-6d9d94b29bd8 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:43.793831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300fd14-d68c-4858-8f5a-6d9d94b29bd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:43.795798 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300fd14-d68c-4858-8f5a-6d9d94b29bd8 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:43.935540 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1143/2306] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:56:43 2022] POST /identity/v3/users => generated 295 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:56:43.938959 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66a5d5-47b7-4d80-bb80-fa8ac68ec415 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:43.955529 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66a5d5-47b7-4d80-bb80-fa8ac68ec415 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:43.960189 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a50279-83eb-4870-917f-ade8a62d1035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:43.960833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a50279-83eb-4870-917f-ade8a62d1035 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:43.971492 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a50279-83eb-4870-917f-ade8a62d1035 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:43.971492 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a50279-83eb-4870-917f-ade8a62d1035 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:43.971492 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a50279-83eb-4870-917f-ade8a62d1035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:43.971492 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a50279-83eb-4870-917f-ade8a62d1035 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:43.975323 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1164/2307] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:56:43.979608 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300fd14-d68c-4858-8f5a-6d9d94b29bd8 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:43.989988 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300fd14-d68c-4858-8f5a-6d9d94b29bd8 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:43.994770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab1766-17e4-460e-bfa9-430acb3ea109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:43.995429 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab1766-17e4-460e-bfa9-430acb3ea109 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:43.995712 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab1766-17e4-460e-bfa9-430acb3ea109 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:43.995988 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab1766-17e4-460e-bfa9-430acb3ea109 admin admin] PATH_INFO: `/v3/projects/66dc3ea932be4cbcac72d19f14f5c8b7/users/1841fb3a324f40c8a063191e87d03ffb/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:44.011347 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dab1766-17e4-460e-bfa9-430acb3ea109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66dc3ea932be4cbcac72d19f14f5c8b7, user_id=1841fb3a324f40c8a063191e87d03ffb, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:44.014042 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dab1766-17e4-460e-bfa9-430acb3ea109 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:44.082913 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1144/2308] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:56:43 2022] PUT /identity/v3/projects/66dc3ea932be4cbcac72d19f14f5c8b7/users/1841fb3a324f40c8a063191e87d03ffb/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:56:44.086946 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a50279-83eb-4870-917f-ade8a62d1035 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:44.098668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a50279-83eb-4870-917f-ade8a62d1035 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:44.181139 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6f23a-ad09-4499-ba74-d585bbf1aedf None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:44.189548 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6f23a-ad09-4499-ba74-d585bbf1aedf None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:44.189548 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6f23a-ad09-4499-ba74-d585bbf1aedf None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:44.189548 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1145/2309] 10.222.0.59 () {54 vars in 921 bytes} [Fri Aug 26 00:56:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:56:44.192960 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef8e84-b8c6-4ad5-987f-75f633bc0989 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:44.193496 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef8e84-b8c6-4ad5-987f-75f633bc0989 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:44.193942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef8e84-b8c6-4ad5-987f-75f633bc0989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:44.206414 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd88a9-04a6-43fb-904e-b214bef4bd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:44.211616 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd88a9-04a6-43fb-904e-b214bef4bd8f admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:44.211616 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd88a9-04a6-43fb-904e-b214bef4bd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:44.211616 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd88a9-04a6-43fb-904e-b214bef4bd8f admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:44.211616 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd88a9-04a6-43fb-904e-b214bef4bd8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:44.211616 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd88a9-04a6-43fb-904e-b214bef4bd8f admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:44.236410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-c6ef8e84-b8c6-4ad5-987f-75f633bc0989 None None] MFA Rules not processed for user `0602593263ca4b05a657d5b7697f932f`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:44.240182 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-c6ef8e84-b8c6-4ad5-987f-75f633bc0989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:44.244277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1165/2310] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:56:44.247877 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd88a9-04a6-43fb-904e-b214bef4bd8f admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:44.274458 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd88a9-04a6-43fb-904e-b214bef4bd8f admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:44.279845 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe063e-2e96-4529-a72b-8a1855f5b796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:44.280956 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe063e-2e96-4529-a72b-8a1855f5b796 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:44.281445 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe063e-2e96-4529-a72b-8a1855f5b796 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:44.282539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe063e-2e96-4529-a72b-8a1855f5b796 admin admin] PATH_INFO: `/v3/projects/66dc3ea932be4cbcac72d19f14f5c8b7/users/1841fb3a324f40c8a063191e87d03ffb/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:44.285101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe063e-2e96-4529-a72b-8a1855f5b796 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66dc3ea932be4cbcac72d19f14f5c8b7, user_id=1841fb3a324f40c8a063191e87d03ffb, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:44.288769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe063e-2e96-4529-a72b-8a1855f5b796 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:44.305747 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1166/2311] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:56:44 2022] PUT /identity/v3/projects/66dc3ea932be4cbcac72d19f14f5c8b7/users/1841fb3a324f40c8a063191e87d03ffb/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:56:44.318556 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47651fbb-3069-46a1-8414-7eb4e1bd98d5 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:44.319207 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47651fbb-3069-46a1-8414-7eb4e1bd98d5 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:44.319645 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47651fbb-3069-46a1-8414-7eb4e1bd98d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:44.385285 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1146/2312] 10.222.0.59 () {58 vars in 1018 bytes} [Fri Aug 26 00:56:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:56:44.392995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-47651fbb-3069-46a1-8414-7eb4e1bd98d5 None None] MFA Rules not processed for user `1841fb3a324f40c8a063191e87d03ffb`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:44.394493 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-47651fbb-3069-46a1-8414-7eb4e1bd98d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:44.396539 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef8e84-b8c6-4ad5-987f-75f633bc0989 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:44.398555 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-c6ef8e84-b8c6-4ad5-987f-75f633bc0989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:44.478232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef8e84-b8c6-4ad5-987f-75f633bc0989 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:44.502014 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1167/2313] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:56:44 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:56:44.506918 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:44.513489 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47651fbb-3069-46a1-8414-7eb4e1bd98d5 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:44.514902 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-47651fbb-3069-46a1-8414-7eb4e1bd98d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:44.559935 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937cfb7-9a65-48c4-a558-882ef4605ef8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abed7ebb0aae4e4693ce08e51a149312', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:44.560947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937cfb7-9a65-48c4-a558-882ef4605ef8 service glance] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:44.561306 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937cfb7-9a65-48c4-a558-882ef4605ef8 service glance] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:44.561657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937cfb7-9a65-48c4-a558-882ef4605ef8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:44.599809 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47651fbb-3069-46a1-8414-7eb4e1bd98d5 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:44.605616 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50d798-0125-4ba9-9c65-d04ad5c428ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:44.606328 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50d798-0125-4ba9-9c65-d04ad5c428ce service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:44.606676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50d798-0125-4ba9-9c65-d04ad5c428ce service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:44.607036 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50d798-0125-4ba9-9c65-d04ad5c428ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:44.623433 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:44.658121 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937cfb7-9a65-48c4-a558-882ef4605ef8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:44.660076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937cfb7-9a65-48c4-a558-882ef4605ef8 service glance] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:44.675190 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1147/2314] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:56:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:56:44.721705 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50d798-0125-4ba9-9c65-d04ad5c428ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:44.723397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50d798-0125-4ba9-9c65-d04ad5c428ce service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:44.734905 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1168/2315] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:56:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:56:44.980450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8da49-a8bb-4c09-9cb8-d6a7e7089159 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:44.981072 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8da49-a8bb-4c09-9cb8-d6a7e7089159 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:44.981072 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8da49-a8bb-4c09-9cb8-d6a7e7089159 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:44.982475 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1148/2316] 10.222.0.59 () {54 vars in 921 bytes} [Fri Aug 26 00:56:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:56:44.989326 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa33bf6-6d40-49f5-84cc-1b5d2244dd42 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:44.989636 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa33bf6-6d40-49f5-84cc-1b5d2244dd42 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:44.989919 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa33bf6-6d40-49f5-84cc-1b5d2244dd42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:45.018002 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-7fa33bf6-6d40-49f5-84cc-1b5d2244dd42 None None] MFA Rules not processed for user `4698a49ed4a04f58a488359a2742e5da`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:45.019391 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-7fa33bf6-6d40-49f5-84cc-1b5d2244dd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:45.092798 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1169/2317] 10.222.0.59 () {58 vars in 1018 bytes} [Fri Aug 26 00:56:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:56:45.105785 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a8da49-a8bb-4c09-9cb8-d6a7e7089159 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:45.107581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-e1a8da49-a8bb-4c09-9cb8-d6a7e7089159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:45.124388 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a8da49-a8bb-4c09-9cb8-d6a7e7089159 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:45.128840 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751fcfde-64ec-458a-b3d1-ac08d26650fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:45.129586 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fcfde-64ec-458a-b3d1-ac08d26650fa service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:45.129934 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fcfde-64ec-458a-b3d1-ac08d26650fa service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:45.130619 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fcfde-64ec-458a-b3d1-ac08d26650fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:45.139529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751fcfde-64ec-458a-b3d1-ac08d26650fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:45.141117 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751fcfde-64ec-458a-b3d1-ac08d26650fa service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:45.166952 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1149/2318] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:56:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 00:56:45.316942 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d0178-2a06-4a98-ab70-fd65ca986f68 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:45.317548 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d0178-2a06-4a98-ab70-fd65ca986f68 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:45.317646 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d0178-2a06-4a98-ab70-fd65ca986f68 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:45.319067 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1170/2319] 10.222.0.59 () {54 vars in 921 bytes} [Fri Aug 26 00:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:56:45.339087 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7459f-6098-4daa-bcec-0e49cfe6a310 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:45.339511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7459f-6098-4daa-bcec-0e49cfe6a310 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:45.339902 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7459f-6098-4daa-bcec-0e49cfe6a310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:45.383216 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-39a7459f-6098-4daa-bcec-0e49cfe6a310 None None] MFA Rules not processed for user `0602593263ca4b05a657d5b7697f932f`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:45.384440 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-39a7459f-6098-4daa-bcec-0e49cfe6a310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:45.415218 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1150/2320] 10.222.0.59 () {58 vars in 1018 bytes} [Fri Aug 26 00:56:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:56:45.416007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d0178-2a06-4a98-ab70-fd65ca986f68 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:45.418133 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-129d0178-2a06-4a98-ab70-fd65ca986f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:45.455482 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d0178-2a06-4a98-ab70-fd65ca986f68 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:45.463506 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46013466-ae60-4afa-9b37-77a33a4390ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:45.466565 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46013466-ae60-4afa-9b37-77a33a4390ff service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:45.466565 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46013466-ae60-4afa-9b37-77a33a4390ff service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:45.466565 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46013466-ae60-4afa-9b37-77a33a4390ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:45.477063 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46013466-ae60-4afa-9b37-77a33a4390ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:45.478934 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46013466-ae60-4afa-9b37-77a33a4390ff service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:45.499045 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1171/2321] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:56:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:56:47.134922 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7459f-6098-4daa-bcec-0e49cfe6a310 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:47.136501 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-39a7459f-6098-4daa-bcec-0e49cfe6a310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:47.163845 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7459f-6098-4daa-bcec-0e49cfe6a310 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:47.174563 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a13d6-80a3-40d6-a881-b4532383626f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:47.175540 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a13d6-80a3-40d6-a881-b4532383626f service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:47.175876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a13d6-80a3-40d6-a881-b4532383626f service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:47.197835 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a13d6-80a3-40d6-a881-b4532383626f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:47.400509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a13d6-80a3-40d6-a881-b4532383626f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:47.404348 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a13d6-80a3-40d6-a881-b4532383626f service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:47.471190 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1151/2322] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:56:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:56:49.714884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46013466-ae60-4afa-9b37-77a33a4390ff service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:49.726478 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46013466-ae60-4afa-9b37-77a33a4390ff service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:49.731927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422aa87-f387-440e-89f8-0c395b310db1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:49.733022 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422aa87-f387-440e-89f8-0c395b310db1 service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:49.733428 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422aa87-f387-440e-89f8-0c395b310db1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:49.733831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422aa87-f387-440e-89f8-0c395b310db1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:49.753315 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422aa87-f387-440e-89f8-0c395b310db1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:49.754835 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422aa87-f387-440e-89f8-0c395b310db1 service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:49.770637 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1172/2323] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:56:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:56:52.924928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a13d6-80a3-40d6-a881-b4532383626f service cinder] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:52.927206 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-367a13d6-80a3-40d6-a881-b4532383626f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:52.989112 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:53.031436 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a13d6-80a3-40d6-a881-b4532383626f service cinder] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:53.064570 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96103af5-f4ba-474d-ae9f-df84a04566ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2876f4182f0949039b862f053533fe85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:53.067192 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96103af5-f4ba-474d-ae9f-df84a04566ba service placement] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:53.067559 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96103af5-f4ba-474d-ae9f-df84a04566ba service placement] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:53.069734 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96103af5-f4ba-474d-ae9f-df84a04566ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:53.073491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-96103af5-f4ba-474d-ae9f-df84a04566ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:53.101873 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96103af5-f4ba-474d-ae9f-df84a04566ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:53.103497 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96103af5-f4ba-474d-ae9f-df84a04566ba service placement] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:53.117340 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1152/2324] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:56:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:56:53.808036 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422aa87-f387-440e-89f8-0c395b310db1 service cinder] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:53.819785 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422aa87-f387-440e-89f8-0c395b310db1 service cinder] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:53.823867 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e387787-ebc3-491c-a43e-c903c888d6aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:53.824664 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e387787-ebc3-491c-a43e-c903c888d6aa service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:53.824942 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e387787-ebc3-491c-a43e-c903c888d6aa service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:53.825414 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e387787-ebc3-491c-a43e-c903c888d6aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:53.828334 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-1e387787-ebc3-491c-a43e-c903c888d6aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:53.926884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e387787-ebc3-491c-a43e-c903c888d6aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:53.928343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e387787-ebc3-491c-a43e-c903c888d6aa service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:53.941073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1173/2325] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:56:53 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:56:54.983075 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96103af5-f4ba-474d-ae9f-df84a04566ba service placement] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:55.111921 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96103af5-f4ba-474d-ae9f-df84a04566ba service placement] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:55.118776 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c514db1-7d46-46be-b8f6-788afcb35ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:55.119515 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c514db1-7d46-46be-b8f6-788afcb35ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:55.120243 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c514db1-7d46-46be-b8f6-788afcb35ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:55.121020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c514db1-7d46-46be-b8f6-788afcb35ed8 admin admin] PATH_INFO: `/v3/users/cc6c20be27a14a08808e10fc843afa82` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:55.122757 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c514db1-7d46-46be-b8f6-788afcb35ed8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc6c20be27a14a08808e10fc843afa82)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:55.125640 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c514db1-7d46-46be-b8f6-788afcb35ed8 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:55.378086 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-6c514db1-7d46-46be-b8f6-788afcb35ed8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc6c20be27a14a08808e10fc843afa82'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:55.394884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-6c514db1-7d46-46be-b8f6-788afcb35ed8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc6c20be27a14a08808e10fc843afa82'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:55.474970 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-6c514db1-7d46-46be-b8f6-788afcb35ed8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc6c20be27a14a08808e10fc843afa82'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:55.475592 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-6c514db1-7d46-46be-b8f6-788afcb35ed8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc6c20be27a14a08808e10fc843afa82'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:55.476270 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-6c514db1-7d46-46be-b8f6-788afcb35ed8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc6c20be27a14a08808e10fc843afa82'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:55.516525 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1153/2326] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:56:54 2022] DELETE /identity/v3/users/cc6c20be27a14a08808e10fc843afa82 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:56:55.523368 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d058dc4-cc97-4bd9-a09e-a79204ef0e9c None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:55.523913 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d058dc4-cc97-4bd9-a09e-a79204ef0e9c None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:55.523913 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d058dc4-cc97-4bd9-a09e-a79204ef0e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:55.549981 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-1d058dc4-cc97-4bd9-a09e-a79204ef0e9c None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:56:55.552440 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-1d058dc4-cc97-4bd9-a09e-a79204ef0e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:55.596195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:55.712526 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1174/2327] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:56:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:56:55.730325 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c514db1-7d46-46be-b8f6-788afcb35ed8 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:55.732205 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-6c514db1-7d46-46be-b8f6-788afcb35ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:55.852790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:56:55.880374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c514db1-7d46-46be-b8f6-788afcb35ed8 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:55.893200 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d30d1-8151-401f-9abf-5c41c0557c5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:55.893863 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d30d1-8151-401f-9abf-5c41c0557c5b service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:55.894202 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d30d1-8151-401f-9abf-5c41c0557c5b service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:55.894590 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d30d1-8151-401f-9abf-5c41c0557c5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:55.917424 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d30d1-8151-401f-9abf-5c41c0557c5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:55.921427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d30d1-8151-401f-9abf-5c41c0557c5b service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:55.942256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1154/2328] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:56:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:56:57.482904 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d058dc4-cc97-4bd9-a09e-a79204ef0e9c None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:57.490825 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-1d058dc4-cc97-4bd9-a09e-a79204ef0e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:56:57.524286 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d058dc4-cc97-4bd9-a09e-a79204ef0e9c None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:57.534158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689402d-d5e1-41ec-b015-ce2cd6cbce0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:57.535243 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689402d-d5e1-41ec-b015-ce2cd6cbce0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:57.535764 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689402d-d5e1-41ec-b015-ce2cd6cbce0d admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:57.536292 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689402d-d5e1-41ec-b015-ce2cd6cbce0d admin admin] PATH_INFO: `/v3/projects/8b67076aed8a4fd59717b7706b1be4b4` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:57.538135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689402d-d5e1-41ec-b015-ce2cd6cbce0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b67076aed8a4fd59717b7706b1be4b4)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:57.541333 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689402d-d5e1-41ec-b015-ce2cd6cbce0d admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:57.785828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-0689402d-d5e1-41ec-b015-ce2cd6cbce0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b67076aed8a4fd59717b7706b1be4b4', 'request_id': 'req-0689402d-d5e1-41ec-b015-ce2cd6cbce0d'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:57.881127 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-0689402d-d5e1-41ec-b015-ce2cd6cbce0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b67076aed8a4fd59717b7706b1be4b4', 'request_id': 'req-0689402d-d5e1-41ec-b015-ce2cd6cbce0d'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:57.889682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-0689402d-d5e1-41ec-b015-ce2cd6cbce0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b67076aed8a4fd59717b7706b1be4b4', 'request_id': 'req-0689402d-d5e1-41ec-b015-ce2cd6cbce0d'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:57.915732 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-0689402d-d5e1-41ec-b015-ce2cd6cbce0d admin admin] The token cache is being invalidate because project 8b67076aed8a4fd59717b7706b1be4b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:56:57.916124 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-0689402d-d5e1-41ec-b015-ce2cd6cbce0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:57.917416 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-0689402d-d5e1-41ec-b015-ce2cd6cbce0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:57.923985 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1175/2329] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:56:57 2022] DELETE /identity/v3/projects/8b67076aed8a4fd59717b7706b1be4b4 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:56:57.924629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d30d1-8151-401f-9abf-5c41c0557c5b service neutron] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:56:57.936552 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d30d1-8151-401f-9abf-5c41c0557c5b service neutron] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:56:58.016417 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6bc4b-6a69-4b23-8212-078f8da5a2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:56:58.017458 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6bc4b-6a69-4b23-8212-078f8da5a2b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:56:58.018007 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6bc4b-6a69-4b23-8212-078f8da5a2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:56:58.018675 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6bc4b-6a69-4b23-8212-078f8da5a2b8 admin admin] PATH_INFO: `/v3/users/0e1b1621c6844d82ac73bfdc7887c5ee` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:56:58.020543 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6bc4b-6a69-4b23-8212-078f8da5a2b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e1b1621c6844d82ac73bfdc7887c5ee)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:56:58.023138 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6bc4b-6a69-4b23-8212-078f8da5a2b8 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:56:58.587009 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-b0d6bc4b-6a69-4b23-8212-078f8da5a2b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e1b1621c6844d82ac73bfdc7887c5ee'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:58.608454 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-b0d6bc4b-6a69-4b23-8212-078f8da5a2b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e1b1621c6844d82ac73bfdc7887c5ee'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:58.697566 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-b0d6bc4b-6a69-4b23-8212-078f8da5a2b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e1b1621c6844d82ac73bfdc7887c5ee'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:58.698287 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-b0d6bc4b-6a69-4b23-8212-078f8da5a2b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e1b1621c6844d82ac73bfdc7887c5ee'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:58.698903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-b0d6bc4b-6a69-4b23-8212-078f8da5a2b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e1b1621c6844d82ac73bfdc7887c5ee'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:56:58.732023 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1155/2330] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:56:57 2022] DELETE /identity/v3/users/0e1b1621c6844d82ac73bfdc7887c5ee => generated 0 bytes in 809 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:57:00.043861 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689402d-d5e1-41ec-b015-ce2cd6cbce0d admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:00.045735 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-0689402d-d5e1-41ec-b015-ce2cd6cbce0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:00.062744 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689402d-d5e1-41ec-b015-ce2cd6cbce0d admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:00.149565 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb83f6-220b-4dff-9f6a-9249496ff03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:00.150745 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb83f6-220b-4dff-9f6a-9249496ff03e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:00.150972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb83f6-220b-4dff-9f6a-9249496ff03e admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:00.151671 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb83f6-220b-4dff-9f6a-9249496ff03e admin admin] PATH_INFO: `/v3/projects/254a371abdc34c90bcf25d7231e728f2` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:00.153089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb83f6-220b-4dff-9f6a-9249496ff03e admin admin] RBAC: Authorizing `identity:delete_project(project_id=254a371abdc34c90bcf25d7231e728f2)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:00.155374 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb83f6-220b-4dff-9f6a-9249496ff03e admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:00.311405 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-30eb83f6-220b-4dff-9f6a-9249496ff03e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '254a371abdc34c90bcf25d7231e728f2', 'request_id': 'req-30eb83f6-220b-4dff-9f6a-9249496ff03e'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:00.314394 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:57:00.478377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-30eb83f6-220b-4dff-9f6a-9249496ff03e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '254a371abdc34c90bcf25d7231e728f2', 'request_id': 'req-30eb83f6-220b-4dff-9f6a-9249496ff03e'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:00.486581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-30eb83f6-220b-4dff-9f6a-9249496ff03e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '254a371abdc34c90bcf25d7231e728f2', 'request_id': 'req-30eb83f6-220b-4dff-9f6a-9249496ff03e'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:00.502398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-30eb83f6-220b-4dff-9f6a-9249496ff03e admin admin] The token cache is being invalidate because project 254a371abdc34c90bcf25d7231e728f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:57:00.502790 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-30eb83f6-220b-4dff-9f6a-9249496ff03e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:00.503458 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-30eb83f6-220b-4dff-9f6a-9249496ff03e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:00.504999 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1176/2331] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:57:00 2022] DELETE /identity/v3/projects/254a371abdc34c90bcf25d7231e728f2 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:57:00.509026 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6bc4b-6a69-4b23-8212-078f8da5a2b8 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:00.510634 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-b0d6bc4b-6a69-4b23-8212-078f8da5a2b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:00.524522 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6bc4b-6a69-4b23-8212-078f8da5a2b8 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:00.595035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befad345-1f5b-4c12-93fa-b454d216799b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:00.595672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befad345-1f5b-4c12-93fa-b454d216799b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:00.595950 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befad345-1f5b-4c12-93fa-b454d216799b admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:00.596302 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befad345-1f5b-4c12-93fa-b454d216799b admin admin] PATH_INFO: `/v3/users/1841fb3a324f40c8a063191e87d03ffb` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:00.597395 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befad345-1f5b-4c12-93fa-b454d216799b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1841fb3a324f40c8a063191e87d03ffb)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:00.599311 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befad345-1f5b-4c12-93fa-b454d216799b admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:00.604084 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:57:00.881976 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-befad345-1f5b-4c12-93fa-b454d216799b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1841fb3a324f40c8a063191e87d03ffb'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:00.894029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-befad345-1f5b-4c12-93fa-b454d216799b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1841fb3a324f40c8a063191e87d03ffb'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:00.977244 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-befad345-1f5b-4c12-93fa-b454d216799b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1841fb3a324f40c8a063191e87d03ffb'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:00.977956 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-befad345-1f5b-4c12-93fa-b454d216799b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1841fb3a324f40c8a063191e87d03ffb'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:00.978570 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-befad345-1f5b-4c12-93fa-b454d216799b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1841fb3a324f40c8a063191e87d03ffb'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:00.995058 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1156/2332] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:57:00 2022] DELETE /identity/v3/users/1841fb3a324f40c8a063191e87d03ffb => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:57:01.029262 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc562fa-af99-4098-a310-bb1763f4382b None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:01.029608 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc562fa-af99-4098-a310-bb1763f4382b None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:01.029888 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc562fa-af99-4098-a310-bb1763f4382b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:01.050216 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-3cc562fa-af99-4098-a310-bb1763f4382b None None] MFA Rules not processed for user `bbd9bbb083ab4f5e84e135ec5103cdae`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:57:01.051547 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-3cc562fa-af99-4098-a310-bb1763f4382b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:01.113981 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1177/2333] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Aug 26 00:57:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:57:01.126715 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befad345-1f5b-4c12-93fa-b454d216799b admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:01.128685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-befad345-1f5b-4c12-93fa-b454d216799b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:01.206966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befad345-1f5b-4c12-93fa-b454d216799b admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:01.212696 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d92a6-e9fa-49da-86f9-7f81de3392c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f700502671446c797a566f07398a9fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:01.213320 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d92a6-e9fa-49da-86f9-7f81de3392c2 service swift] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:01.213604 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d92a6-e9fa-49da-86f9-7f81de3392c2 service swift] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:01.213881 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d92a6-e9fa-49da-86f9-7f81de3392c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:01.224420 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d92a6-e9fa-49da-86f9-7f81de3392c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:01.226453 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d92a6-e9fa-49da-86f9-7f81de3392c2 service swift] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:01.236000 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1157/2334] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Aug 26 00:57:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 00:57:01.443033 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc562fa-af99-4098-a310-bb1763f4382b None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:01.444804 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-3cc562fa-af99-4098-a310-bb1763f4382b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:01.619287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc562fa-af99-4098-a310-bb1763f4382b None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:01.627110 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec224c53-f3fd-4099-bd91-0640c3071061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:01.628018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec224c53-f3fd-4099-bd91-0640c3071061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:01.628503 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec224c53-f3fd-4099-bd91-0640c3071061 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:01.628904 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec224c53-f3fd-4099-bd91-0640c3071061 admin admin] PATH_INFO: `/v3/projects/66dc3ea932be4cbcac72d19f14f5c8b7` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:01.631320 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec224c53-f3fd-4099-bd91-0640c3071061 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66dc3ea932be4cbcac72d19f14f5c8b7)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:01.634355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec224c53-f3fd-4099-bd91-0640c3071061 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:01.756378 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ec224c53-f3fd-4099-bd91-0640c3071061 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66dc3ea932be4cbcac72d19f14f5c8b7', 'request_id': 'req-ec224c53-f3fd-4099-bd91-0640c3071061'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:01.876346 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ec224c53-f3fd-4099-bd91-0640c3071061 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66dc3ea932be4cbcac72d19f14f5c8b7', 'request_id': 'req-ec224c53-f3fd-4099-bd91-0640c3071061'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:01.883005 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ec224c53-f3fd-4099-bd91-0640c3071061 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66dc3ea932be4cbcac72d19f14f5c8b7', 'request_id': 'req-ec224c53-f3fd-4099-bd91-0640c3071061'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:01.899094 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ec224c53-f3fd-4099-bd91-0640c3071061 admin admin] The token cache is being invalidate because project 66dc3ea932be4cbcac72d19f14f5c8b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:57:01.899835 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ec224c53-f3fd-4099-bd91-0640c3071061 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:01.900958 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ec224c53-f3fd-4099-bd91-0640c3071061 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:01.903557 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1178/2335] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:57:01 2022] DELETE /identity/v3/projects/66dc3ea932be4cbcac72d19f14f5c8b7 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:57:01.920708 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f000be-ae3f-4e95-aff9-55756659a0a8 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:01.921612 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f000be-ae3f-4e95-aff9-55756659a0a8 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:01.922075 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f000be-ae3f-4e95-aff9-55756659a0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:01.950088 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-02f000be-ae3f-4e95-aff9-55756659a0a8 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:57:01.952090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-02f000be-ae3f-4e95-aff9-55756659a0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:02.051283 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1158/2336] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:57:02.053564 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196b4c0-1101-4707-9ebf-984a055cd500 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:02.054372 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196b4c0-1101-4707-9ebf-984a055cd500 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:02.054816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196b4c0-1101-4707-9ebf-984a055cd500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:02.087655 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-3196b4c0-1101-4707-9ebf-984a055cd500 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:57:02.089082 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-3196b4c0-1101-4707-9ebf-984a055cd500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:02.098010 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1179/2337] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:57:02.102425 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f000be-ae3f-4e95-aff9-55756659a0a8 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:02.128126 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f000be-ae3f-4e95-aff9-55756659a0a8 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:02.130413 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea6a45-f57c-445e-9f54-687eefcdec49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:02.130934 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea6a45-f57c-445e-9f54-687eefcdec49 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:02.131200 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea6a45-f57c-445e-9f54-687eefcdec49 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:02.131473 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea6a45-f57c-445e-9f54-687eefcdec49 admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:02.132446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cea6a45-f57c-445e-9f54-687eefcdec49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:57:02.132446 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cea6a45-f57c-445e-9f54-687eefcdec49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:02.134101 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cea6a45-f57c-445e-9f54-687eefcdec49 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:02.145002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:57:02.148785 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1159/2338] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:57:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:57:02.153145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196b4c0-1101-4707-9ebf-984a055cd500 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:02.162964 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196b4c0-1101-4707-9ebf-984a055cd500 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:02.167820 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3077e7e-a105-4202-840c-1b485d4e3a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:02.168438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3077e7e-a105-4202-840c-1b485d4e3a58 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:02.168880 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3077e7e-a105-4202-840c-1b485d4e3a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:02.169154 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3077e7e-a105-4202-840c-1b485d4e3a58 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:02.169786 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3077e7e-a105-4202-840c-1b485d4e3a58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:02.171756 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3077e7e-a105-4202-840c-1b485d4e3a58 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:02.178950 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:57:02.271686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1180/2339] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:57:02 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:57:02.275751 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea6a45-f57c-445e-9f54-687eefcdec49 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:02.286220 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea6a45-f57c-445e-9f54-687eefcdec49 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:02.352536 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c74e4-6b6d-40a5-ace8-7ea252f8b173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:02.353165 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c74e4-6b6d-40a5-ace8-7ea252f8b173 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:02.353603 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c74e4-6b6d-40a5-ace8-7ea252f8b173 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:02.353894 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c74e4-6b6d-40a5-ace8-7ea252f8b173 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:02.354542 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677c74e4-6b6d-40a5-ace8-7ea252f8b173 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:02.356578 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677c74e4-6b6d-40a5-ace8-7ea252f8b173 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:02.481746 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1160/2340] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:57:02 2022] POST /identity/v3/users => generated 287 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:57:02.487018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3077e7e-a105-4202-840c-1b485d4e3a58 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:02.499517 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3077e7e-a105-4202-840c-1b485d4e3a58 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:02.504428 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d84976-974d-4d5e-bcbc-fa6bb88ad257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:02.507376 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d84976-974d-4d5e-bcbc-fa6bb88ad257 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:02.507376 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d84976-974d-4d5e-bcbc-fa6bb88ad257 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:02.507376 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d84976-974d-4d5e-bcbc-fa6bb88ad257 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:02.507376 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d84976-974d-4d5e-bcbc-fa6bb88ad257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:02.507947 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d84976-974d-4d5e-bcbc-fa6bb88ad257 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:02.519027 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1181/2341] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:57:02.522971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c74e4-6b6d-40a5-ace8-7ea252f8b173 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:02.533600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c74e4-6b6d-40a5-ace8-7ea252f8b173 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:02.538694 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74e6ac-cb44-4d7d-a3c4-8ef88cdd45c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:02.539349 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74e6ac-cb44-4d7d-a3c4-8ef88cdd45c5 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:02.539666 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74e6ac-cb44-4d7d-a3c4-8ef88cdd45c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:02.539975 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74e6ac-cb44-4d7d-a3c4-8ef88cdd45c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:02.540714 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74e6ac-cb44-4d7d-a3c4-8ef88cdd45c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:02.542530 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74e6ac-cb44-4d7d-a3c4-8ef88cdd45c5 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:02.555140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1161/2342] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:57:02.559000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d84976-974d-4d5e-bcbc-fa6bb88ad257 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:02.569879 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d84976-974d-4d5e-bcbc-fa6bb88ad257 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:02.576207 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e27d33-b194-47bd-86a7-c1a51ec3ac0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:02.576900 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e27d33-b194-47bd-86a7-c1a51ec3ac0d admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:02.583345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e27d33-b194-47bd-86a7-c1a51ec3ac0d admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:02.583760 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e27d33-b194-47bd-86a7-c1a51ec3ac0d admin admin] PATH_INFO: `/v3/projects/7b6fe7a5a87c4740b32ee5e1cc1b7458/users/5c4fa4658a714288862f8a4d328f0f9c/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:02.607002 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e27d33-b194-47bd-86a7-c1a51ec3ac0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b6fe7a5a87c4740b32ee5e1cc1b7458, user_id=5c4fa4658a714288862f8a4d328f0f9c, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:02.610463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e27d33-b194-47bd-86a7-c1a51ec3ac0d admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:02.684831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1182/2343] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:57:02 2022] PUT /identity/v3/projects/7b6fe7a5a87c4740b32ee5e1cc1b7458/users/5c4fa4658a714288862f8a4d328f0f9c/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:57:02.696245 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb835f12-f58c-44a2-a858-9b736a4cacee None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:02.696575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb835f12-f58c-44a2-a858-9b736a4cacee None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:02.696876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb835f12-f58c-44a2-a858-9b736a4cacee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:02.730656 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-fb835f12-f58c-44a2-a858-9b736a4cacee None None] MFA Rules not processed for user `5c4fa4658a714288862f8a4d328f0f9c`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:57:02.732016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-fb835f12-f58c-44a2-a858-9b736a4cacee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:02.799401 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1162/2344] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:57:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:57:02.810536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e27d33-b194-47bd-86a7-c1a51ec3ac0d admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:02.812539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-32e27d33-b194-47bd-86a7-c1a51ec3ac0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:02.884973 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e27d33-b194-47bd-86a7-c1a51ec3ac0d admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:02.888899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed497b-1d10-48c1-9452-e9d5aa7294a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:02.889431 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed497b-1d10-48c1-9452-e9d5aa7294a0 service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:02.889808 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed497b-1d10-48c1-9452-e9d5aa7294a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:02.889974 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed497b-1d10-48c1-9452-e9d5aa7294a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:02.902459 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed497b-1d10-48c1-9452-e9d5aa7294a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:02.904791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed497b-1d10-48c1-9452-e9d5aa7294a0 service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:02.918429 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1183/2345] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:57:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:57:04.804758 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040027d2-8e58-412c-a05a-19bd3c3e6d1d None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:04.805398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040027d2-8e58-412c-a05a-19bd3c3e6d1d None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:04.805398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040027d2-8e58-412c-a05a-19bd3c3e6d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:04.827312 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-040027d2-8e58-412c-a05a-19bd3c3e6d1d None None] MFA Rules not processed for user `5c4fa4658a714288862f8a4d328f0f9c`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:57:04.829365 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-040027d2-8e58-412c-a05a-19bd3c3e6d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:04.837288 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1163/2346] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:57:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:57:08.563389 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0efce-c9aa-4a52-b0db-c8dc2a65f3f5 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:08.563967 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0efce-c9aa-4a52-b0db-c8dc2a65f3f5 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:08.563967 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0efce-c9aa-4a52-b0db-c8dc2a65f3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:08.584154 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-e2d0efce-c9aa-4a52-b0db-c8dc2a65f3f5 None None] MFA Rules not processed for user `bbd9bbb083ab4f5e84e135ec5103cdae`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:57:08.585523 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-e2d0efce-c9aa-4a52-b0db-c8dc2a65f3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:08.647897 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1184/2347] 10.222.0.59 () {58 vars in 967 bytes} [Fri Aug 26 00:57:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:57:08.663930 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040027d2-8e58-412c-a05a-19bd3c3e6d1d None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:08.665582 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-040027d2-8e58-412c-a05a-19bd3c3e6d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:08.736331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:57:08.741070 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040027d2-8e58-412c-a05a-19bd3c3e6d1d None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:08.745608 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab998f4-c593-4663-ac16-33d78542d4c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f700502671446c797a566f07398a9fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:08.746241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab998f4-c593-4663-ac16-33d78542d4c6 service swift] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:08.746521 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab998f4-c593-4663-ac16-33d78542d4c6 service swift] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:08.746797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab998f4-c593-4663-ac16-33d78542d4c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:08.755717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab998f4-c593-4663-ac16-33d78542d4c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:08.757498 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab998f4-c593-4663-ac16-33d78542d4c6 service swift] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:08.765798 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1164/2348] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Aug 26 00:57:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 00:57:12.178261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0efce-c9aa-4a52-b0db-c8dc2a65f3f5 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:12.179791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-e2d0efce-c9aa-4a52-b0db-c8dc2a65f3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:12.246902 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0efce-c9aa-4a52-b0db-c8dc2a65f3f5 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:12.252045 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e221b7-dc95-49fa-8aeb-c9133a0ac92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:12.252709 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e221b7-dc95-49fa-8aeb-c9133a0ac92e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:12.252990 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e221b7-dc95-49fa-8aeb-c9133a0ac92e admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:12.253260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e221b7-dc95-49fa-8aeb-c9133a0ac92e admin admin] PATH_INFO: `/v3/users/2f93774232c1479aa3e856bffb352140` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:12.254384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e221b7-dc95-49fa-8aeb-c9133a0ac92e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f93774232c1479aa3e856bffb352140)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:12.256298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e221b7-dc95-49fa-8aeb-c9133a0ac92e admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:12.257778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:57:12.448401 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-43e221b7-dc95-49fa-8aeb-c9133a0ac92e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f93774232c1479aa3e856bffb352140'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:12.454624 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-43e221b7-dc95-49fa-8aeb-c9133a0ac92e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f93774232c1479aa3e856bffb352140'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:12.461609 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-43e221b7-dc95-49fa-8aeb-c9133a0ac92e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f93774232c1479aa3e856bffb352140'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:12.525717 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-43e221b7-dc95-49fa-8aeb-c9133a0ac92e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f93774232c1479aa3e856bffb352140'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:12.526389 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-43e221b7-dc95-49fa-8aeb-c9133a0ac92e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f93774232c1479aa3e856bffb352140'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:12.539282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1185/2349] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:57:12 2022] DELETE /identity/v3/users/2f93774232c1479aa3e856bffb352140 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:57:12.867207 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab998f4-c593-4663-ac16-33d78542d4c6 service swift] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:12.877301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab998f4-c593-4663-ac16-33d78542d4c6 service swift] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:12.935813 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e5514-5450-4384-ba28-ffe4a3135593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:12.936451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e5514-5450-4384-ba28-ffe4a3135593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:12.936770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e5514-5450-4384-ba28-ffe4a3135593 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:12.937044 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e5514-5450-4384-ba28-ffe4a3135593 admin admin] PATH_INFO: `/v3/projects/9b49ff217de14a2aae05d524128dec49` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:12.938363 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e5514-5450-4384-ba28-ffe4a3135593 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b49ff217de14a2aae05d524128dec49)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:12.940291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e5514-5450-4384-ba28-ffe4a3135593 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:13.073504 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-333e5514-5450-4384-ba28-ffe4a3135593 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b49ff217de14a2aae05d524128dec49', 'request_id': 'req-333e5514-5450-4384-ba28-ffe4a3135593'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:13.126393 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-333e5514-5450-4384-ba28-ffe4a3135593 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b49ff217de14a2aae05d524128dec49', 'request_id': 'req-333e5514-5450-4384-ba28-ffe4a3135593'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:13.133536 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-333e5514-5450-4384-ba28-ffe4a3135593 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b49ff217de14a2aae05d524128dec49', 'request_id': 'req-333e5514-5450-4384-ba28-ffe4a3135593'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:13.149427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-333e5514-5450-4384-ba28-ffe4a3135593 admin admin] The token cache is being invalidate because project 9b49ff217de14a2aae05d524128dec49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:57:13.149719 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-333e5514-5450-4384-ba28-ffe4a3135593 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:13.150407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-333e5514-5450-4384-ba28-ffe4a3135593 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:13.151895 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1165/2350] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:57:12 2022] DELETE /identity/v3/projects/9b49ff217de14a2aae05d524128dec49 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:57:13.157543 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fc878-1c63-413a-8266-f219cef510c9 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:13.157831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fc878-1c63-413a-8266-f219cef510c9 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:13.161329 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fc878-1c63-413a-8266-f219cef510c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:13.177972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-f96fc878-1c63-413a-8266-f219cef510c9 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:57:13.179191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f96fc878-1c63-413a-8266-f219cef510c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:13.240029 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1186/2351] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:57:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:57:13.248214 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841aa5bc-3516-493d-a56b-68e6a6164398 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:13.248528 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841aa5bc-3516-493d-a56b-68e6a6164398 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:13.248807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841aa5bc-3516-493d-a56b-68e6a6164398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:13.269725 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-841aa5bc-3516-493d-a56b-68e6a6164398 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:57:13.270923 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-841aa5bc-3516-493d-a56b-68e6a6164398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:13.278606 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1166/2352] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:57:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:57:13.283345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96fc878-1c63-413a-8266-f219cef510c9 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:13.293542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96fc878-1c63-413a-8266-f219cef510c9 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:13.298171 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f44cd5-43c0-49f4-a62f-b2f781c4ac5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:13.298952 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f44cd5-43c0-49f4-a62f-b2f781c4ac5d admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:13.299161 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f44cd5-43c0-49f4-a62f-b2f781c4ac5d admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:13.299329 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f44cd5-43c0-49f4-a62f-b2f781c4ac5d admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:13.300009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f44cd5-43c0-49f4-a62f-b2f781c4ac5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:57:13.300314 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f44cd5-43c0-49f4-a62f-b2f781c4ac5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:13.302189 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f44cd5-43c0-49f4-a62f-b2f781c4ac5d admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:13.314030 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1187/2353] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:57:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:57:13.318467 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841aa5bc-3516-493d-a56b-68e6a6164398 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:13.328017 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841aa5bc-3516-493d-a56b-68e6a6164398 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:13.333235 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39439c4f-950c-4661-99d1-a728d4ea586e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:13.333836 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39439c4f-950c-4661-99d1-a728d4ea586e admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:13.334114 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39439c4f-950c-4661-99d1-a728d4ea586e admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:13.334384 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39439c4f-950c-4661-99d1-a728d4ea586e admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:13.335032 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39439c4f-950c-4661-99d1-a728d4ea586e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:13.336983 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39439c4f-950c-4661-99d1-a728d4ea586e admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:13.431439 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1167/2354] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:57:13 2022] POST /identity/v3/projects => generated 367 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:57:13.435452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f44cd5-43c0-49f4-a62f-b2f781c4ac5d admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:13.446274 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f44cd5-43c0-49f4-a62f-b2f781c4ac5d admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:13.504720 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385ee95-324e-4646-96a3-3aee944727e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:13.505332 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385ee95-324e-4646-96a3-3aee944727e7 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:13.505614 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385ee95-324e-4646-96a3-3aee944727e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:13.505888 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385ee95-324e-4646-96a3-3aee944727e7 admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:13.506554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385ee95-324e-4646-96a3-3aee944727e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:13.508542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385ee95-324e-4646-96a3-3aee944727e7 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:13.520035 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:57:13.605779 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1188/2355] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:57:13 2022] POST /identity/v3/users => generated 284 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:57:13.609927 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39439c4f-950c-4661-99d1-a728d4ea586e admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:13.620014 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39439c4f-950c-4661-99d1-a728d4ea586e admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:13.624614 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956adcf-5410-47b5-8974-08ef54acef24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:13.625728 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956adcf-5410-47b5-8974-08ef54acef24 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:13.625978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956adcf-5410-47b5-8974-08ef54acef24 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:13.626251 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956adcf-5410-47b5-8974-08ef54acef24 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:13.626894 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0956adcf-5410-47b5-8974-08ef54acef24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:13.628510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0956adcf-5410-47b5-8974-08ef54acef24 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:13.638218 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1168/2356] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:57:13.642847 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385ee95-324e-4646-96a3-3aee944727e7 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:13.655500 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385ee95-324e-4646-96a3-3aee944727e7 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:13.658420 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ea700-0c97-45f9-bebd-382647541497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:13.659020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ea700-0c97-45f9-bebd-382647541497 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:13.659344 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ea700-0c97-45f9-bebd-382647541497 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:13.659596 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ea700-0c97-45f9-bebd-382647541497 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:13.660237 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9ea700-0c97-45f9-bebd-382647541497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:13.661877 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9ea700-0c97-45f9-bebd-382647541497 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:13.674253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1189/2357] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:57:13.677504 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956adcf-5410-47b5-8974-08ef54acef24 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:13.687306 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956adcf-5410-47b5-8974-08ef54acef24 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:13.692165 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608426d-e7ad-468d-8de2-b8764177fba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:13.692849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608426d-e7ad-468d-8de2-b8764177fba9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:13.693134 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608426d-e7ad-468d-8de2-b8764177fba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:13.694004 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608426d-e7ad-468d-8de2-b8764177fba9 admin admin] PATH_INFO: `/v3/projects/7c44c4b73a16423587685ddcee1f9f5e/users/0a52f801006e4995852c8754ebcf6791/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:13.708808 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608426d-e7ad-468d-8de2-b8764177fba9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c44c4b73a16423587685ddcee1f9f5e, user_id=0a52f801006e4995852c8754ebcf6791, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:13.711452 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608426d-e7ad-468d-8de2-b8764177fba9 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:13.784030 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1169/2358] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:57:13 2022] PUT /identity/v3/projects/7c44c4b73a16423587685ddcee1f9f5e/users/0a52f801006e4995852c8754ebcf6791/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:57:13.794834 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d439c5-15a4-4f9a-ae7b-c78071f80899 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:13.795128 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d439c5-15a4-4f9a-ae7b-c78071f80899 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:13.795403 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d439c5-15a4-4f9a-ae7b-c78071f80899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:13.824073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-f8d439c5-15a4-4f9a-ae7b-c78071f80899 None None] MFA Rules not processed for user `0a52f801006e4995852c8754ebcf6791`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:57:13.827430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f8d439c5-15a4-4f9a-ae7b-c78071f80899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:13.896284 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1190/2359] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:57:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:57:13.909337 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608426d-e7ad-468d-8de2-b8764177fba9 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:13.911149 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-c608426d-e7ad-468d-8de2-b8764177fba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:13.932134 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:57:13.970021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608426d-e7ad-468d-8de2-b8764177fba9 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:13.973725 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700f760-ea67-4c3a-bfb3-0da3b0d56a8b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abed7ebb0aae4e4693ce08e51a149312', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:13.974337 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700f760-ea67-4c3a-bfb3-0da3b0d56a8b service glance] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:13.974623 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700f760-ea67-4c3a-bfb3-0da3b0d56a8b service glance] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:13.974900 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700f760-ea67-4c3a-bfb3-0da3b0d56a8b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:13.983413 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700f760-ea67-4c3a-bfb3-0da3b0d56a8b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:13.984910 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700f760-ea67-4c3a-bfb3-0da3b0d56a8b service glance] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:13.993661 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1170/2360] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:57:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:57:22.884255 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d439c5-15a4-4f9a-ae7b-c78071f80899 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:22.885775 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f8d439c5-15a4-4f9a-ae7b-c78071f80899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:22.953491 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d439c5-15a4-4f9a-ae7b-c78071f80899 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:22.957976 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a2380-1251-4d37-81bd-20e99b4e4ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:22.958635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a2380-1251-4d37-81bd-20e99b4e4ef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:22.958921 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a2380-1251-4d37-81bd-20e99b4e4ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:22.959195 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a2380-1251-4d37-81bd-20e99b4e4ef4 admin admin] PATH_INFO: `/v3/users/92c33a46fce541de835acb37b0b4057d` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:22.960310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a2380-1251-4d37-81bd-20e99b4e4ef4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92c33a46fce541de835acb37b0b4057d)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:22.962329 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a2380-1251-4d37-81bd-20e99b4e4ef4 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:23.106083 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-690a2380-1251-4d37-81bd-20e99b4e4ef4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92c33a46fce541de835acb37b0b4057d'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:23.106749 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-690a2380-1251-4d37-81bd-20e99b4e4ef4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92c33a46fce541de835acb37b0b4057d'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:23.119494 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-690a2380-1251-4d37-81bd-20e99b4e4ef4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92c33a46fce541de835acb37b0b4057d'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:23.183030 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-690a2380-1251-4d37-81bd-20e99b4e4ef4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92c33a46fce541de835acb37b0b4057d'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:23.183758 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-690a2380-1251-4d37-81bd-20e99b4e4ef4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92c33a46fce541de835acb37b0b4057d'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:23.190204 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:57:23.200943 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1191/2361] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:57:22 2022] DELETE /identity/v3/users/92c33a46fce541de835acb37b0b4057d => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 00:57:23.206383 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ae1f9-403b-474b-950b-2a719f7ff5ac None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:23.207060 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ae1f9-403b-474b-950b-2a719f7ff5ac None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:23.207060 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ae1f9-403b-474b-950b-2a719f7ff5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:23.229483 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-774ae1f9-403b-474b-950b-2a719f7ff5ac None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:57:23.230868 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-774ae1f9-403b-474b-950b-2a719f7ff5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:23.435252 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1171/2362] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:57:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:57:23.447132 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a2380-1251-4d37-81bd-20e99b4e4ef4 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:23.448466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-690a2380-1251-4d37-81bd-20e99b4e4ef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:23.526638 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a2380-1251-4d37-81bd-20e99b4e4ef4 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:23.531060 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff80eb3-f0ca-496c-8a19-70512fa47695 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:23.531685 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff80eb3-f0ca-496c-8a19-70512fa47695 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:23.531971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff80eb3-f0ca-496c-8a19-70512fa47695 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:23.532242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff80eb3-f0ca-496c-8a19-70512fa47695 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:23.548619 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff80eb3-f0ca-496c-8a19-70512fa47695 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:23.550088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff80eb3-f0ca-496c-8a19-70512fa47695 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:23.560735 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1192/2363] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:57:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:57:24.021543 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ae1f9-403b-474b-950b-2a719f7ff5ac None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:24.023004 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-774ae1f9-403b-474b-950b-2a719f7ff5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:24.045497 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ae1f9-403b-474b-950b-2a719f7ff5ac None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:24.051887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c94f43b-721d-4d3e-bff5-c04feba35c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:24.052860 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94f43b-721d-4d3e-bff5-c04feba35c3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:24.053142 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94f43b-721d-4d3e-bff5-c04feba35c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:24.053471 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94f43b-721d-4d3e-bff5-c04feba35c3b admin admin] PATH_INFO: `/v3/projects/457bcaaea73f487e97bb76fc3e1f6870` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:24.054806 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c94f43b-721d-4d3e-bff5-c04feba35c3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=457bcaaea73f487e97bb76fc3e1f6870)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:24.059516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c94f43b-721d-4d3e-bff5-c04feba35c3b admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:24.185331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-4c94f43b-721d-4d3e-bff5-c04feba35c3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '457bcaaea73f487e97bb76fc3e1f6870', 'request_id': 'req-4c94f43b-721d-4d3e-bff5-c04feba35c3b'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:24.235180 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-4c94f43b-721d-4d3e-bff5-c04feba35c3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '457bcaaea73f487e97bb76fc3e1f6870', 'request_id': 'req-4c94f43b-721d-4d3e-bff5-c04feba35c3b'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:24.242302 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-4c94f43b-721d-4d3e-bff5-c04feba35c3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '457bcaaea73f487e97bb76fc3e1f6870', 'request_id': 'req-4c94f43b-721d-4d3e-bff5-c04feba35c3b'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:24.253029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-4c94f43b-721d-4d3e-bff5-c04feba35c3b admin admin] The token cache is being invalidate because project 457bcaaea73f487e97bb76fc3e1f6870 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:57:24.253341 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-4c94f43b-721d-4d3e-bff5-c04feba35c3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:24.254042 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-4c94f43b-721d-4d3e-bff5-c04feba35c3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:24.255516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1172/2364] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:57:24 2022] DELETE /identity/v3/projects/457bcaaea73f487e97bb76fc3e1f6870 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:57:24.260508 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c25b4-be10-4187-b821-a34f47cda6c7 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:24.260989 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c25b4-be10-4187-b821-a34f47cda6c7 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:24.261112 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c25b4-be10-4187-b821-a34f47cda6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:24.286880 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-829c25b4-be10-4187-b821-a34f47cda6c7 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:57:24.286880 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-829c25b4-be10-4187-b821-a34f47cda6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:24.342395 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1193/2365] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:57:24.350856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6b0d6-622d-4dd6-a628-e964dd706022 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:24.351137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6b0d6-622d-4dd6-a628-e964dd706022 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:24.351412 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6b0d6-622d-4dd6-a628-e964dd706022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:24.372657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-b1a6b0d6-622d-4dd6-a628-e964dd706022 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:57:24.373871 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-b1a6b0d6-622d-4dd6-a628-e964dd706022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:24.381842 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1173/2366] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:57:24.386024 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c25b4-be10-4187-b821-a34f47cda6c7 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:24.396228 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c25b4-be10-4187-b821-a34f47cda6c7 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:24.400950 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154f14b-84d9-41c1-a3f5-5b74717eb8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:24.401556 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154f14b-84d9-41c1-a3f5-5b74717eb8f1 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:24.401840 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154f14b-84d9-41c1-a3f5-5b74717eb8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:24.402114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154f14b-84d9-41c1-a3f5-5b74717eb8f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:24.402762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154f14b-84d9-41c1-a3f5-5b74717eb8f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:57:24.403097 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154f14b-84d9-41c1-a3f5-5b74717eb8f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:24.404730 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154f14b-84d9-41c1-a3f5-5b74717eb8f1 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:24.417694 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1194/2367] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:57:24.421628 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6b0d6-622d-4dd6-a628-e964dd706022 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:24.431006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6b0d6-622d-4dd6-a628-e964dd706022 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:24.444993 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6adb43e-1421-4933-bda9-caa86688b9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:24.445397 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6adb43e-1421-4933-bda9-caa86688b9a4 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:24.446078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6adb43e-1421-4933-bda9-caa86688b9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:24.446363 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6adb43e-1421-4933-bda9-caa86688b9a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:24.447004 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6adb43e-1421-4933-bda9-caa86688b9a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:24.449026 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6adb43e-1421-4933-bda9-caa86688b9a4 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:24.550094 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1174/2368] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:57:24 2022] POST /identity/v3/projects => generated 391 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:57:24.553242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154f14b-84d9-41c1-a3f5-5b74717eb8f1 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:24.564648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154f14b-84d9-41c1-a3f5-5b74717eb8f1 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:24.632180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c0de5-5714-4765-afbf-25ee37cfd7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:24.632958 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c0de5-5714-4765-afbf-25ee37cfd7e1 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:24.633304 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c0de5-5714-4765-afbf-25ee37cfd7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:24.633673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c0de5-5714-4765-afbf-25ee37cfd7e1 admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:24.634403 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c0de5-5714-4765-afbf-25ee37cfd7e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:24.636444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c0de5-5714-4765-afbf-25ee37cfd7e1 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:24.704549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1195/2369] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:57:24 2022] POST /identity/v3/users => generated 296 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:57:24.707945 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6adb43e-1421-4933-bda9-caa86688b9a4 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:24.720753 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6adb43e-1421-4933-bda9-caa86688b9a4 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:24.726835 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f610221-db32-4a95-8e16-1c942c2cfb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:24.727671 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f610221-db32-4a95-8e16-1c942c2cfb6d admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:24.727996 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f610221-db32-4a95-8e16-1c942c2cfb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:24.728272 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f610221-db32-4a95-8e16-1c942c2cfb6d admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:24.733244 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f610221-db32-4a95-8e16-1c942c2cfb6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:24.733244 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f610221-db32-4a95-8e16-1c942c2cfb6d admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:24.742438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1175/2370] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:57:24.746758 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c0de5-5714-4765-afbf-25ee37cfd7e1 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:24.757924 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c0de5-5714-4765-afbf-25ee37cfd7e1 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:24.762697 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe9228-e8f1-478f-ac43-fc4e16d5e131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:24.763307 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe9228-e8f1-478f-ac43-fc4e16d5e131 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:24.763583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe9228-e8f1-478f-ac43-fc4e16d5e131 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:24.763850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe9228-e8f1-478f-ac43-fc4e16d5e131 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:24.764514 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe9228-e8f1-478f-ac43-fc4e16d5e131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:24.766181 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe9228-e8f1-478f-ac43-fc4e16d5e131 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:24.781834 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1196/2371] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:57:24.783267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f610221-db32-4a95-8e16-1c942c2cfb6d admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:24.790494 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:57:24.796508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f610221-db32-4a95-8e16-1c942c2cfb6d admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:24.801221 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c316a-17f0-4723-955a-7efcf576a45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:24.801828 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c316a-17f0-4723-955a-7efcf576a45b admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:24.802108 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c316a-17f0-4723-955a-7efcf576a45b admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:24.802384 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c316a-17f0-4723-955a-7efcf576a45b admin admin] PATH_INFO: `/v3/projects/a0158f1622aa4651adac28233915d940/users/306519c52bef4f85be6f7ee110289f3f/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:24.818447 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c316a-17f0-4723-955a-7efcf576a45b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0158f1622aa4651adac28233915d940, user_id=306519c52bef4f85be6f7ee110289f3f, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:24.821250 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c316a-17f0-4723-955a-7efcf576a45b admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:24.903508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1176/2372] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:57:24 2022] PUT /identity/v3/projects/a0158f1622aa4651adac28233915d940/users/306519c52bef4f85be6f7ee110289f3f/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:57:24.917139 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627080f-335e-4414-a3c0-97d0eab48d15 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:24.917429 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627080f-335e-4414-a3c0-97d0eab48d15 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:24.917703 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627080f-335e-4414-a3c0-97d0eab48d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:24.925560 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:57:24.947817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-3627080f-335e-4414-a3c0-97d0eab48d15 None None] MFA Rules not processed for user `306519c52bef4f85be6f7ee110289f3f`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:57:24.949156 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-3627080f-335e-4414-a3c0-97d0eab48d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:25.022567 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1197/2373] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:57:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:57:25.033888 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c316a-17f0-4723-955a-7efcf576a45b admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:25.035617 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-fa0c316a-17f0-4723-955a-7efcf576a45b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:25.093073 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c316a-17f0-4723-955a-7efcf576a45b admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:25.097347 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1cdf5-751d-4a36-acc7-4b4b66774907 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:25.097955 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1cdf5-751d-4a36-acc7-4b4b66774907 service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:25.098236 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1cdf5-751d-4a36-acc7-4b4b66774907 service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:25.098513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1cdf5-751d-4a36-acc7-4b4b66774907 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:25.107161 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee1cdf5-751d-4a36-acc7-4b4b66774907 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:25.108691 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee1cdf5-751d-4a36-acc7-4b4b66774907 service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:25.118652 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1177/2374] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:57:25 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:57:27.731059 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca921f75-8e0f-4a96-903d-522abe8737d7 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:27.731548 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca921f75-8e0f-4a96-903d-522abe8737d7 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:27.731643 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca921f75-8e0f-4a96-903d-522abe8737d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:27.751532 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-ca921f75-8e0f-4a96-903d-522abe8737d7 None None] MFA Rules not processed for user `bbd9bbb083ab4f5e84e135ec5103cdae`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:57:27.752806 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-ca921f75-8e0f-4a96-903d-522abe8737d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:27.815291 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1198/2375] 10.222.0.59 () {58 vars in 967 bytes} [Fri Aug 26 00:57:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:57:27.836910 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1cdf5-751d-4a36-acc7-4b4b66774907 service cinder] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:27.838291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-8ee1cdf5-751d-4a36-acc7-4b4b66774907 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:27.910991 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1cdf5-751d-4a36-acc7-4b4b66774907 service cinder] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:27.916108 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91404fa2-7cfa-463f-8999-301168631f66 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f700502671446c797a566f07398a9fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:27.916742 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91404fa2-7cfa-463f-8999-301168631f66 service swift] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:27.917023 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91404fa2-7cfa-463f-8999-301168631f66 service swift] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:27.917302 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91404fa2-7cfa-463f-8999-301168631f66 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:27.926600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91404fa2-7cfa-463f-8999-301168631f66 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:27.928199 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91404fa2-7cfa-463f-8999-301168631f66 service swift] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:27.936118 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1178/2376] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Aug 26 00:57:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 00:57:37.154320 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bd08c-9c75-40b8-afca-e0daebd9102f None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:37.154785 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bd08c-9c75-40b8-afca-e0daebd9102f None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:37.154880 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bd08c-9c75-40b8-afca-e0daebd9102f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:37.175145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-4e6bd08c-9c75-40b8-afca-e0daebd9102f None None] MFA Rules not processed for user `bbd9bbb083ab4f5e84e135ec5103cdae`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:57:37.176455 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-4e6bd08c-9c75-40b8-afca-e0daebd9102f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:37.183717 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1199/2377] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Aug 26 00:57:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:57:37.198332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91404fa2-7cfa-463f-8999-301168631f66 service swift] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:37.209424 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91404fa2-7cfa-463f-8999-301168631f66 service swift] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:37.214429 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6ab4a-22fd-41be-b751-30d9ea3cac53 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f700502671446c797a566f07398a9fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:37.215017 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6ab4a-22fd-41be-b751-30d9ea3cac53 service swift] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:37.215294 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6ab4a-22fd-41be-b751-30d9ea3cac53 service swift] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:37.215562 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6ab4a-22fd-41be-b751-30d9ea3cac53 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:37.224031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6ab4a-22fd-41be-b751-30d9ea3cac53 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:37.225717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6ab4a-22fd-41be-b751-30d9ea3cac53 service swift] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:37.233784 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1179/2378] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Aug 26 00:57:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 00:57:38.155615 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ff0e6-7891-490f-b32e-4aff3ea0344e None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:38.156863 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ff0e6-7891-490f-b32e-4aff3ea0344e None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:38.156947 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ff0e6-7891-490f-b32e-4aff3ea0344e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:38.180992 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-bf1ff0e6-7891-490f-b32e-4aff3ea0344e None None] MFA Rules not processed for user `bbd9bbb083ab4f5e84e135ec5103cdae`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:57:38.182194 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-bf1ff0e6-7891-490f-b32e-4aff3ea0344e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:38.189620 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1200/2379] 10.222.0.59 () {58 vars in 967 bytes} [Fri Aug 26 00:57:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:57:38.202328 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6ab4a-22fd-41be-b751-30d9ea3cac53 service swift] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:38.213565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6ab4a-22fd-41be-b751-30d9ea3cac53 service swift] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:38.218631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182138a6-93b5-46cd-9119-befa016523b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f700502671446c797a566f07398a9fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:38.219917 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182138a6-93b5-46cd-9119-befa016523b7 service swift] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:38.220202 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182138a6-93b5-46cd-9119-befa016523b7 service swift] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:38.220482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182138a6-93b5-46cd-9119-befa016523b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:38.228593 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182138a6-93b5-46cd-9119-befa016523b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:38.230155 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182138a6-93b5-46cd-9119-befa016523b7 service swift] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:38.238397 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1180/2380] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Aug 26 00:57:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 00:57:39.361339 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252248c-7b55-48e1-b2f4-823e6cb8659a None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:39.361874 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252248c-7b55-48e1-b2f4-823e6cb8659a None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:39.361874 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252248c-7b55-48e1-b2f4-823e6cb8659a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:39.381059 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-a252248c-7b55-48e1-b2f4-823e6cb8659a None None] MFA Rules not processed for user `bbd9bbb083ab4f5e84e135ec5103cdae`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:57:39.382258 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a252248c-7b55-48e1-b2f4-823e6cb8659a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:39.389498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1201/2381] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Aug 26 00:57:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:57:39.401730 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182138a6-93b5-46cd-9119-befa016523b7 service swift] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:39.412923 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182138a6-93b5-46cd-9119-befa016523b7 service swift] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:39.418202 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760e163-0cec-4190-8fc9-aa09dc4ac05e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f700502671446c797a566f07398a9fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:39.418800 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760e163-0cec-4190-8fc9-aa09dc4ac05e service swift] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:39.419079 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760e163-0cec-4190-8fc9-aa09dc4ac05e service swift] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:39.419351 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760e163-0cec-4190-8fc9-aa09dc4ac05e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:39.427320 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760e163-0cec-4190-8fc9-aa09dc4ac05e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:39.428920 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760e163-0cec-4190-8fc9-aa09dc4ac05e service swift] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:39.437055 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1181/2382] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Aug 26 00:57:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 00:57:39.995035 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f5a1a-1a46-4fe9-af2b-c0c6d79c45bb None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:39.995384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f5a1a-1a46-4fe9-af2b-c0c6d79c45bb None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:39.995721 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f5a1a-1a46-4fe9-af2b-c0c6d79c45bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:40.015366 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-8a4f5a1a-1a46-4fe9-af2b-c0c6d79c45bb None None] MFA Rules not processed for user `bbd9bbb083ab4f5e84e135ec5103cdae`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:57:40.017125 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-8a4f5a1a-1a46-4fe9-af2b-c0c6d79c45bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:40.024037 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1202/2383] 10.222.0.59 () {58 vars in 967 bytes} [Fri Aug 26 00:57:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:57:40.036576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760e163-0cec-4190-8fc9-aa09dc4ac05e service swift] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:40.047722 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760e163-0cec-4190-8fc9-aa09dc4ac05e service swift] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:40.053037 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b7a22-ac8b-454d-9377-034d5e6286e2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f700502671446c797a566f07398a9fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:40.053631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b7a22-ac8b-454d-9377-034d5e6286e2 service swift] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:40.053910 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b7a22-ac8b-454d-9377-034d5e6286e2 service swift] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:40.054178 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b7a22-ac8b-454d-9377-034d5e6286e2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:40.062049 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b7a22-ac8b-454d-9377-034d5e6286e2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:40.064181 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b7a22-ac8b-454d-9377-034d5e6286e2 service swift] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:40.071987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1182/2384] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Aug 26 00:57:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 00:57:45.941277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f5a1a-1a46-4fe9-af2b-c0c6d79c45bb None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:45.942877 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-8a4f5a1a-1a46-4fe9-af2b-c0c6d79c45bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:46.009298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f5a1a-1a46-4fe9-af2b-c0c6d79c45bb None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:46.014028 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f64d6-2a91-43a9-b2a3-bbcd9f4dad8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:46.014666 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f64d6-2a91-43a9-b2a3-bbcd9f4dad8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:46.014950 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f64d6-2a91-43a9-b2a3-bbcd9f4dad8a admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:46.015227 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f64d6-2a91-43a9-b2a3-bbcd9f4dad8a admin admin] PATH_INFO: `/v3/users/5c4fa4658a714288862f8a4d328f0f9c` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:46.017060 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f64d6-2a91-43a9-b2a3-bbcd9f4dad8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c4fa4658a714288862f8a4d328f0f9c)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:46.019032 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f64d6-2a91-43a9-b2a3-bbcd9f4dad8a admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:46.031536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:57:46.224619 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-e72f64d6-2a91-43a9-b2a3-bbcd9f4dad8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c4fa4658a714288862f8a4d328f0f9c'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:46.225274 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-e72f64d6-2a91-43a9-b2a3-bbcd9f4dad8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c4fa4658a714288862f8a4d328f0f9c'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:46.237672 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-e72f64d6-2a91-43a9-b2a3-bbcd9f4dad8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c4fa4658a714288862f8a4d328f0f9c'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:46.319856 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-e72f64d6-2a91-43a9-b2a3-bbcd9f4dad8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c4fa4658a714288862f8a4d328f0f9c'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:46.320519 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-e72f64d6-2a91-43a9-b2a3-bbcd9f4dad8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c4fa4658a714288862f8a4d328f0f9c'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:46.333831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1203/2385] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:57:45 2022] DELETE /identity/v3/users/5c4fa4658a714288862f8a4d328f0f9c => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:57:46.338517 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896981c6-0838-467a-a94b-00dc7870379b None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:46.339017 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896981c6-0838-467a-a94b-00dc7870379b None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:46.339121 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896981c6-0838-467a-a94b-00dc7870379b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:46.358301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-896981c6-0838-467a-a94b-00dc7870379b None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:57:46.359474 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-896981c6-0838-467a-a94b-00dc7870379b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:46.372127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:57:46.418312 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1183/2386] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:57:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:57:46.428134 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f64d6-2a91-43a9-b2a3-bbcd9f4dad8a admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:46.429621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-e72f64d6-2a91-43a9-b2a3-bbcd9f4dad8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:46.494191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f64d6-2a91-43a9-b2a3-bbcd9f4dad8a admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:46.497940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22db9dc-a205-4b74-a96f-12153907f5b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:46.499256 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22db9dc-a205-4b74-a96f-12153907f5b1 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:46.499256 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22db9dc-a205-4b74-a96f-12153907f5b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:46.499457 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22db9dc-a205-4b74-a96f-12153907f5b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:46.509434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22db9dc-a205-4b74-a96f-12153907f5b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:46.510968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22db9dc-a205-4b74-a96f-12153907f5b1 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:46.521758 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1204/2387] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:57:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:57:46.761894 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896981c6-0838-467a-a94b-00dc7870379b None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:46.763327 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-896981c6-0838-467a-a94b-00dc7870379b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:46.775899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896981c6-0838-467a-a94b-00dc7870379b None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:46.780962 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28586619-0f1b-4ce9-b967-19351324bb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:46.781597 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28586619-0f1b-4ce9-b967-19351324bb08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:46.781877 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28586619-0f1b-4ce9-b967-19351324bb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:46.782145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28586619-0f1b-4ce9-b967-19351324bb08 admin admin] PATH_INFO: `/v3/projects/7b6fe7a5a87c4740b32ee5e1cc1b7458` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:46.783305 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28586619-0f1b-4ce9-b967-19351324bb08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b6fe7a5a87c4740b32ee5e1cc1b7458)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:46.785403 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28586619-0f1b-4ce9-b967-19351324bb08 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:46.897235 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-28586619-0f1b-4ce9-b967-19351324bb08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b6fe7a5a87c4740b32ee5e1cc1b7458', 'request_id': 'req-28586619-0f1b-4ce9-b967-19351324bb08'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:46.948092 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-28586619-0f1b-4ce9-b967-19351324bb08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b6fe7a5a87c4740b32ee5e1cc1b7458', 'request_id': 'req-28586619-0f1b-4ce9-b967-19351324bb08'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:46.954756 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-28586619-0f1b-4ce9-b967-19351324bb08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b6fe7a5a87c4740b32ee5e1cc1b7458', 'request_id': 'req-28586619-0f1b-4ce9-b967-19351324bb08'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:46.965141 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-28586619-0f1b-4ce9-b967-19351324bb08 admin admin] The token cache is being invalidate because project 7b6fe7a5a87c4740b32ee5e1cc1b7458 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:57:46.965432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-28586619-0f1b-4ce9-b967-19351324bb08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:46.966073 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-28586619-0f1b-4ce9-b967-19351324bb08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:46.967549 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1184/2388] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:57:46 2022] DELETE /identity/v3/projects/7b6fe7a5a87c4740b32ee5e1cc1b7458 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:57:49.997199 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22db9dc-a205-4b74-a96f-12153907f5b1 service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:49.998626 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-b22db9dc-a205-4b74-a96f-12153907f5b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:50.011401 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22db9dc-a205-4b74-a96f-12153907f5b1 service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:50.068521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3525d006-0858-40a7-aaf6-eaa56fe803f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:50.069163 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525d006-0858-40a7-aaf6-eaa56fe803f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:50.069443 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525d006-0858-40a7-aaf6-eaa56fe803f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:50.069714 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525d006-0858-40a7-aaf6-eaa56fe803f5 admin admin] PATH_INFO: `/v3/users/306519c52bef4f85be6f7ee110289f3f` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:50.070831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3525d006-0858-40a7-aaf6-eaa56fe803f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=306519c52bef4f85be6f7ee110289f3f)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:50.072876 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3525d006-0858-40a7-aaf6-eaa56fe803f5 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:50.245170 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:57:50.254949 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-3525d006-0858-40a7-aaf6-eaa56fe803f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '306519c52bef4f85be6f7ee110289f3f'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:50.255616 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-3525d006-0858-40a7-aaf6-eaa56fe803f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '306519c52bef4f85be6f7ee110289f3f'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:50.267890 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-3525d006-0858-40a7-aaf6-eaa56fe803f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '306519c52bef4f85be6f7ee110289f3f'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:50.342494 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-3525d006-0858-40a7-aaf6-eaa56fe803f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '306519c52bef4f85be6f7ee110289f3f'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:50.343041 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-3525d006-0858-40a7-aaf6-eaa56fe803f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '306519c52bef4f85be6f7ee110289f3f'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:50.355750 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1205/2389] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:57:49 2022] DELETE /identity/v3/users/306519c52bef4f85be6f7ee110289f3f => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:57:50.360608 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f825d4-9858-42f2-a8d7-cda86742846a None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:50.361078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f825d4-9858-42f2-a8d7-cda86742846a None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:50.361182 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f825d4-9858-42f2-a8d7-cda86742846a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:50.380074 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-85f825d4-9858-42f2-a8d7-cda86742846a None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:57:50.381296 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-85f825d4-9858-42f2-a8d7-cda86742846a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:50.442003 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1185/2390] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:57:50.451733 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3525d006-0858-40a7-aaf6-eaa56fe803f5 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:50.453155 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-3525d006-0858-40a7-aaf6-eaa56fe803f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:50.510113 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3525d006-0858-40a7-aaf6-eaa56fe803f5 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:50.513671 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3a60e-0692-4c3b-9a0e-bc35a7c322c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:50.514266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3a60e-0692-4c3b-9a0e-bc35a7c322c7 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:50.514540 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3a60e-0692-4c3b-9a0e-bc35a7c322c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:50.514806 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3a60e-0692-4c3b-9a0e-bc35a7c322c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:50.524345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3a60e-0692-4c3b-9a0e-bc35a7c322c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:50.525857 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3a60e-0692-4c3b-9a0e-bc35a7c322c7 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:50.537607 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1206/2391] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:57:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:57:50.910606 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f825d4-9858-42f2-a8d7-cda86742846a None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:50.912135 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-85f825d4-9858-42f2-a8d7-cda86742846a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:50.925018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f825d4-9858-42f2-a8d7-cda86742846a None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:50.929697 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965bf64-8b45-4645-aa4a-d759e7684410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:50.930334 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965bf64-8b45-4645-aa4a-d759e7684410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:50.930672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965bf64-8b45-4645-aa4a-d759e7684410 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:50.930951 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965bf64-8b45-4645-aa4a-d759e7684410 admin admin] PATH_INFO: `/v3/projects/a0158f1622aa4651adac28233915d940` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:50.932112 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965bf64-8b45-4645-aa4a-d759e7684410 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0158f1622aa4651adac28233915d940)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:50.934251 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965bf64-8b45-4645-aa4a-d759e7684410 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:51.022864 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:57:51.029826 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0965bf64-8b45-4645-aa4a-d759e7684410 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0158f1622aa4651adac28233915d940', 'request_id': 'req-0965bf64-8b45-4645-aa4a-d759e7684410'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:51.108402 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0965bf64-8b45-4645-aa4a-d759e7684410 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0158f1622aa4651adac28233915d940', 'request_id': 'req-0965bf64-8b45-4645-aa4a-d759e7684410'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:51.115270 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0965bf64-8b45-4645-aa4a-d759e7684410 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0158f1622aa4651adac28233915d940', 'request_id': 'req-0965bf64-8b45-4645-aa4a-d759e7684410'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:51.125858 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0965bf64-8b45-4645-aa4a-d759e7684410 admin admin] The token cache is being invalidate because project a0158f1622aa4651adac28233915d940 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:57:51.126150 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0965bf64-8b45-4645-aa4a-d759e7684410 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:51.127281 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0965bf64-8b45-4645-aa4a-d759e7684410 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:51.128791 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1186/2392] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:57:50 2022] DELETE /identity/v3/projects/a0158f1622aa4651adac28233915d940 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:57:59.100943 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3a60e-0692-4c3b-9a0e-bc35a7c322c7 service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:59.102390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-16d3a60e-0692-4c3b-9a0e-bc35a7c322c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:59.115595 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3a60e-0692-4c3b-9a0e-bc35a7c322c7 service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:59.171053 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751aa83-2442-4339-82c1-e39c30a2d243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:59.171684 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751aa83-2442-4339-82c1-e39c30a2d243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:59.171967 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751aa83-2442-4339-82c1-e39c30a2d243 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:59.172244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751aa83-2442-4339-82c1-e39c30a2d243 admin admin] PATH_INFO: `/v3/users/0a52f801006e4995852c8754ebcf6791` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:59.173414 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751aa83-2442-4339-82c1-e39c30a2d243 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a52f801006e4995852c8754ebcf6791)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:59.175975 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751aa83-2442-4339-82c1-e39c30a2d243 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:59.355701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-9751aa83-2442-4339-82c1-e39c30a2d243 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a52f801006e4995852c8754ebcf6791'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:59.356377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-9751aa83-2442-4339-82c1-e39c30a2d243 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a52f801006e4995852c8754ebcf6791'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:59.363862 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:57:59.369035 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-9751aa83-2442-4339-82c1-e39c30a2d243 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a52f801006e4995852c8754ebcf6791'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:59.456835 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-9751aa83-2442-4339-82c1-e39c30a2d243 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a52f801006e4995852c8754ebcf6791'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:59.457462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-9751aa83-2442-4339-82c1-e39c30a2d243 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a52f801006e4995852c8754ebcf6791'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:57:59.470429 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1207/2393] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:57:59 2022] DELETE /identity/v3/users/0a52f801006e4995852c8754ebcf6791 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:57:59.475530 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2f6ed-9387-4375-8a4e-53bb5f4531ef None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:59.476012 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2f6ed-9387-4375-8a4e-53bb5f4531ef None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:59.476133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2f6ed-9387-4375-8a4e-53bb5f4531ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:59.495920 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-19b2f6ed-9387-4375-8a4e-53bb5f4531ef None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:57:59.497148 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-19b2f6ed-9387-4375-8a4e-53bb5f4531ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:59.560005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1187/2394] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:57:59.570712 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751aa83-2442-4339-82c1-e39c30a2d243 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:57:59.572110 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-9751aa83-2442-4339-82c1-e39c30a2d243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:57:59.629904 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751aa83-2442-4339-82c1-e39c30a2d243 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:57:59.633426 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885571f-ece3-4ce7-97a2-96ce8e016ff5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:57:59.634025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885571f-ece3-4ce7-97a2-96ce8e016ff5 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:57:59.634297 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885571f-ece3-4ce7-97a2-96ce8e016ff5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:57:59.634570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885571f-ece3-4ce7-97a2-96ce8e016ff5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:57:59.643354 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8885571f-ece3-4ce7-97a2-96ce8e016ff5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:57:59.644837 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8885571f-ece3-4ce7-97a2-96ce8e016ff5 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:57:59.654012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1208/2395] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:57:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:58:00.053410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2f6ed-9387-4375-8a4e-53bb5f4531ef None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:58:00.056133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-19b2f6ed-9387-4375-8a4e-53bb5f4531ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:58:00.068506 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2f6ed-9387-4375-8a4e-53bb5f4531ef None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:58:00.073136 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38feca-18a7-42ff-b8d1-a1482642104d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:58:00.073766 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38feca-18a7-42ff-b8d1-a1482642104d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:58:00.074043 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38feca-18a7-42ff-b8d1-a1482642104d admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:58:00.074844 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38feca-18a7-42ff-b8d1-a1482642104d admin admin] PATH_INFO: `/v3/projects/7c44c4b73a16423587685ddcee1f9f5e` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:58:00.076151 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b38feca-18a7-42ff-b8d1-a1482642104d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c44c4b73a16423587685ddcee1f9f5e)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:58:00.083133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b38feca-18a7-42ff-b8d1-a1482642104d admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:58:00.176934 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-8b38feca-18a7-42ff-b8d1-a1482642104d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c44c4b73a16423587685ddcee1f9f5e', 'request_id': 'req-8b38feca-18a7-42ff-b8d1-a1482642104d'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:58:00.227858 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-8b38feca-18a7-42ff-b8d1-a1482642104d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c44c4b73a16423587685ddcee1f9f5e', 'request_id': 'req-8b38feca-18a7-42ff-b8d1-a1482642104d'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:58:00.234065 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-8b38feca-18a7-42ff-b8d1-a1482642104d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c44c4b73a16423587685ddcee1f9f5e', 'request_id': 'req-8b38feca-18a7-42ff-b8d1-a1482642104d'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:58:00.244682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-8b38feca-18a7-42ff-b8d1-a1482642104d admin admin] The token cache is being invalidate because project 7c44c4b73a16423587685ddcee1f9f5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:58:00.245004 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-8b38feca-18a7-42ff-b8d1-a1482642104d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:58:00.245596 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-8b38feca-18a7-42ff-b8d1-a1482642104d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:58:00.247131 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1188/2396] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:58:00 2022] DELETE /identity/v3/projects/7c44c4b73a16423587685ddcee1f9f5e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:58:00.252056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294dca52-79db-4388-a604-92d6081655c3 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:58:00.252566 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294dca52-79db-4388-a604-92d6081655c3 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:58:00.252667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294dca52-79db-4388-a604-92d6081655c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:58:00.273776 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-294dca52-79db-4388-a604-92d6081655c3 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:58:00.274973 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-294dca52-79db-4388-a604-92d6081655c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:58:00.332027 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1209/2397] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:58:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:58:00.339841 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d771e7d-caca-46e5-b295-bce7485fa536 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:58:00.340123 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d771e7d-caca-46e5-b295-bce7485fa536 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:58:00.340390 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d771e7d-caca-46e5-b295-bce7485fa536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:58:00.359480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-3d771e7d-caca-46e5-b295-bce7485fa536 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:58:00.360700 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-3d771e7d-caca-46e5-b295-bce7485fa536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:58:00.368904 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1189/2398] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:58:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:58:00.373196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294dca52-79db-4388-a604-92d6081655c3 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:58:00.383679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294dca52-79db-4388-a604-92d6081655c3 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:58:00.388248 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cacae79-1457-4952-8520-e8eb3c972d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:58:00.389067 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacae79-1457-4952-8520-e8eb3c972d4b admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:58:00.389353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacae79-1457-4952-8520-e8eb3c972d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:58:00.389657 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacae79-1457-4952-8520-e8eb3c972d4b admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:58:00.390323 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cacae79-1457-4952-8520-e8eb3c972d4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 00:58:00.390634 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cacae79-1457-4952-8520-e8eb3c972d4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:58:00.392232 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cacae79-1457-4952-8520-e8eb3c972d4b admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:58:00.403843 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1210/2399] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 00:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 00:58:00.409596 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d771e7d-caca-46e5-b295-bce7485fa536 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:58:00.419011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d771e7d-caca-46e5-b295-bce7485fa536 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:58:00.423722 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da318b-7a51-4322-9aad-4268c3c8db83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:58:00.424312 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da318b-7a51-4322-9aad-4268c3c8db83 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:58:00.424626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da318b-7a51-4322-9aad-4268c3c8db83 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:58:00.424899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da318b-7a51-4322-9aad-4268c3c8db83 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:58:00.425522 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da318b-7a51-4322-9aad-4268c3c8db83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:58:00.427466 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da318b-7a51-4322-9aad-4268c3c8db83 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:58:00.537765 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1190/2400] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 00:58:00 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 00:58:00.542180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cacae79-1457-4952-8520-e8eb3c972d4b admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:58:00.553513 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cacae79-1457-4952-8520-e8eb3c972d4b admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:58:00.611995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a09a38-a3be-4813-8a78-5f6d50492b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:58:00.612643 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a09a38-a3be-4813-8a78-5f6d50492b95 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:58:00.612925 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a09a38-a3be-4813-8a78-5f6d50492b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:58:00.613197 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a09a38-a3be-4813-8a78-5f6d50492b95 admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:58:00.613850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a09a38-a3be-4813-8a78-5f6d50492b95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:58:00.616416 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a09a38-a3be-4813-8a78-5f6d50492b95 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:58:00.626227 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:58:00.714376 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1211/2401] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 00:58:00 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 00:58:00.718596 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da318b-7a51-4322-9aad-4268c3c8db83 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:58:00.729290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da318b-7a51-4322-9aad-4268c3c8db83 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:58:00.733993 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c258b4-f867-495a-ae6a-b11a98786740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:58:00.734591 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c258b4-f867-495a-ae6a-b11a98786740 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:58:00.734887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c258b4-f867-495a-ae6a-b11a98786740 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:58:00.735161 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c258b4-f867-495a-ae6a-b11a98786740 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:58:00.735794 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c258b4-f867-495a-ae6a-b11a98786740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:58:00.737407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c258b4-f867-495a-ae6a-b11a98786740 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:58:00.747582 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1191/2402] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:58:00.752148 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a09a38-a3be-4813-8a78-5f6d50492b95 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:58:00.762122 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a09a38-a3be-4813-8a78-5f6d50492b95 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:58:00.766768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8f56f-8734-402b-b457-d020c63dd0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:58:00.767370 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8f56f-8734-402b-b457-d020c63dd0d9 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:58:00.767652 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8f56f-8734-402b-b457-d020c63dd0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:58:00.772559 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8f56f-8734-402b-b457-d020c63dd0d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:58:00.772559 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f8f56f-8734-402b-b457-d020c63dd0d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:58:00.772559 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f8f56f-8734-402b-b457-d020c63dd0d9 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:58:00.780728 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1212/2403] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 00:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 00:58:00.784905 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c258b4-f867-495a-ae6a-b11a98786740 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:58:00.795736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c258b4-f867-495a-ae6a-b11a98786740 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:58:00.800397 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4f884-89fb-409e-b0ad-2399e0d9bb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:58:00.801085 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4f884-89fb-409e-b0ad-2399e0d9bb2d admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:58:00.806946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4f884-89fb-409e-b0ad-2399e0d9bb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:58:00.806946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4f884-89fb-409e-b0ad-2399e0d9bb2d admin admin] PATH_INFO: `/v3/projects/dd32e075295c478a8cb5f785e8fbcb94/users/eb155e8baa6e400096693f564a389985/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:58:00.810495 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:58:00.816647 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce4f884-89fb-409e-b0ad-2399e0d9bb2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd32e075295c478a8cb5f785e8fbcb94, user_id=eb155e8baa6e400096693f564a389985, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:58:00.819402 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce4f884-89fb-409e-b0ad-2399e0d9bb2d admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:58:00.902516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1192/2404] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 00:58:00 2022] PUT /identity/v3/projects/dd32e075295c478a8cb5f785e8fbcb94/users/eb155e8baa6e400096693f564a389985/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:58:00.913167 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad42e1b-8640-4f32-95e8-a6431f8dd6c7 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:58:00.913714 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad42e1b-8640-4f32-95e8-a6431f8dd6c7 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:58:00.914013 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad42e1b-8640-4f32-95e8-a6431f8dd6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:58:00.941810 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-5ad42e1b-8640-4f32-95e8-a6431f8dd6c7 None None] MFA Rules not processed for user `eb155e8baa6e400096693f564a389985`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:58:00.943000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-5ad42e1b-8640-4f32-95e8-a6431f8dd6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:58:01.005394 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1213/2405] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:58:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:58:01.016535 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4f884-89fb-409e-b0ad-2399e0d9bb2d admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:58:01.018216 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-dce4f884-89fb-409e-b0ad-2399e0d9bb2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:58:01.075846 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4f884-89fb-409e-b0ad-2399e0d9bb2d admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:58:01.079656 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d83c999-66cf-4146-a744-c6fffb825a7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:58:01.080292 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83c999-66cf-4146-a744-c6fffb825a7d service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:58:01.080606 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83c999-66cf-4146-a744-c6fffb825a7d service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:58:01.080885 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83c999-66cf-4146-a744-c6fffb825a7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:58:01.089981 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d83c999-66cf-4146-a744-c6fffb825a7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:58:01.091549 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d83c999-66cf-4146-a744-c6fffb825a7d service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:58:01.100980 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1193/2406] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:58:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:58:04.024533 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad42e1b-8640-4f32-95e8-a6431f8dd6c7 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:58:04.091871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad42e1b-8640-4f32-95e8-a6431f8dd6c7 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:58:04.097274 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5deb8-3e23-49e1-8c49-1a6a5f4106be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:58:04.097918 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5deb8-3e23-49e1-8c49-1a6a5f4106be admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:58:04.098261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5deb8-3e23-49e1-8c49-1a6a5f4106be admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:58:04.098545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5deb8-3e23-49e1-8c49-1a6a5f4106be admin admin] PATH_INFO: `/v3/users/eb155e8baa6e400096693f564a389985` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:58:04.099709 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5deb8-3e23-49e1-8c49-1a6a5f4106be admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb155e8baa6e400096693f564a389985)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:58:04.101733 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5deb8-3e23-49e1-8c49-1a6a5f4106be admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:58:04.283433 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-63d5deb8-3e23-49e1-8c49-1a6a5f4106be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb155e8baa6e400096693f564a389985'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:58:04.284094 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-63d5deb8-3e23-49e1-8c49-1a6a5f4106be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb155e8baa6e400096693f564a389985'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:58:04.296791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-63d5deb8-3e23-49e1-8c49-1a6a5f4106be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb155e8baa6e400096693f564a389985'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:58:04.371264 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-63d5deb8-3e23-49e1-8c49-1a6a5f4106be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb155e8baa6e400096693f564a389985'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:58:04.372000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-63d5deb8-3e23-49e1-8c49-1a6a5f4106be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb155e8baa6e400096693f564a389985'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:58:04.374289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:58:04.385254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1214/2407] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 00:58:04 2022] DELETE /identity/v3/users/eb155e8baa6e400096693f564a389985 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:58:04.389937 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae340609-a8eb-4b58-926e-f46b6ab5669c None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:58:04.390454 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae340609-a8eb-4b58-926e-f46b6ab5669c None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:58:04.390454 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae340609-a8eb-4b58-926e-f46b6ab5669c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:58:04.408803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-ae340609-a8eb-4b58-926e-f46b6ab5669c None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:58:04.409983 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-ae340609-a8eb-4b58-926e-f46b6ab5669c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:58:04.468989 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1194/2408] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:58:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:58:04.479367 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5deb8-3e23-49e1-8c49-1a6a5f4106be admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:58:04.480948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-63d5deb8-3e23-49e1-8c49-1a6a5f4106be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:58:04.539081 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5deb8-3e23-49e1-8c49-1a6a5f4106be admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:58:04.542793 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4903c-1d1b-4015-af72-338b8a6c8541 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:58:04.543404 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4903c-1d1b-4015-af72-338b8a6c8541 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:58:04.543683 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4903c-1d1b-4015-af72-338b8a6c8541 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:58:04.543955 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4903c-1d1b-4015-af72-338b8a6c8541 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:58:04.552947 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4903c-1d1b-4015-af72-338b8a6c8541 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:58:04.554902 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4903c-1d1b-4015-af72-338b8a6c8541 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:58:04.564363 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1215/2409] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:58:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:58:04.949649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae340609-a8eb-4b58-926e-f46b6ab5669c None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:58:04.952300 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-ae340609-a8eb-4b58-926e-f46b6ab5669c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:58:04.965227 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae340609-a8eb-4b58-926e-f46b6ab5669c None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:58:04.971379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e53b35-6d25-4eb1-98e5-4acc6fbb7a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:58:04.972063 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e53b35-6d25-4eb1-98e5-4acc6fbb7a01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:58:04.972343 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e53b35-6d25-4eb1-98e5-4acc6fbb7a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:58:04.972670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e53b35-6d25-4eb1-98e5-4acc6fbb7a01 admin admin] PATH_INFO: `/v3/projects/dd32e075295c478a8cb5f785e8fbcb94` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:58:04.973923 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e53b35-6d25-4eb1-98e5-4acc6fbb7a01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd32e075295c478a8cb5f785e8fbcb94)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:58:04.977899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e53b35-6d25-4eb1-98e5-4acc6fbb7a01 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:58:05.091916 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-01e53b35-6d25-4eb1-98e5-4acc6fbb7a01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd32e075295c478a8cb5f785e8fbcb94', 'request_id': 'req-01e53b35-6d25-4eb1-98e5-4acc6fbb7a01'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:58:05.142662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-01e53b35-6d25-4eb1-98e5-4acc6fbb7a01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd32e075295c478a8cb5f785e8fbcb94', 'request_id': 'req-01e53b35-6d25-4eb1-98e5-4acc6fbb7a01'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:58:05.150034 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-01e53b35-6d25-4eb1-98e5-4acc6fbb7a01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd32e075295c478a8cb5f785e8fbcb94', 'request_id': 'req-01e53b35-6d25-4eb1-98e5-4acc6fbb7a01'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:58:05.160753 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-01e53b35-6d25-4eb1-98e5-4acc6fbb7a01 admin admin] The token cache is being invalidate because project dd32e075295c478a8cb5f785e8fbcb94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 00:58:05.161052 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-01e53b35-6d25-4eb1-98e5-4acc6fbb7a01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:58:05.161692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-01e53b35-6d25-4eb1-98e5-4acc6fbb7a01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 00:58:05.163159 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1195/2410] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 00:58:04 2022] DELETE /identity/v3/projects/dd32e075295c478a8cb5f785e8fbcb94 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 00:58:14.560176 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=30 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:58:15.157829 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:58:26.649754 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf5672-c8f0-4a82-89f7-edfb5ab6b506 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:58:26.649754 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf5672-c8f0-4a82-89f7-edfb5ab6b506 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:58:26.649754 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf5672-c8f0-4a82-89f7-edfb5ab6b506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:58:26.669552 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-80cf5672-c8f0-4a82-89f7-edfb5ab6b506 None None] MFA Rules not processed for user `34240c306dc54f0c8758dbb4b4525376`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:58:26.670836 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-80cf5672-c8f0-4a82-89f7-edfb5ab6b506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:58:26.727711 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1216/2411] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:58:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:58:36.721065 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=25 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:58:39.698825 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e53b35-6d25-4eb1-98e5-4acc6fbb7a01 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:58:39.700260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-01e53b35-6d25-4eb1-98e5-4acc6fbb7a01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:58:39.759920 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e53b35-6d25-4eb1-98e5-4acc6fbb7a01 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:58:39.764166 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082b797-00b3-4478-92b3-fdfe9235f42d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:58:39.764800 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082b797-00b3-4478-92b3-fdfe9235f42d service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:58:39.765076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082b797-00b3-4478-92b3-fdfe9235f42d service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:58:39.765346 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082b797-00b3-4478-92b3-fdfe9235f42d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:58:39.767059 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-b082b797-00b3-4478-92b3-fdfe9235f42d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:58:39.836004 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082b797-00b3-4478-92b3-fdfe9235f42d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:58:39.837476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082b797-00b3-4478-92b3-fdfe9235f42d service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:58:39.848534 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1196/2412] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:58:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:58:40.287462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf5672-c8f0-4a82-89f7-edfb5ab6b506 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:58:40.289157 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-80cf5672-c8f0-4a82-89f7-edfb5ab6b506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:58:40.301850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf5672-c8f0-4a82-89f7-edfb5ab6b506 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:58:40.305393 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8128394-5ce0-45e8-98a0-fe9f97d4cc52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:58:40.306000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8128394-5ce0-45e8-98a0-fe9f97d4cc52 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:58:40.306284 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8128394-5ce0-45e8-98a0-fe9f97d4cc52 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:58:40.306560 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8128394-5ce0-45e8-98a0-fe9f97d4cc52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:58:40.308248 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-c8128394-5ce0-45e8-98a0-fe9f97d4cc52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:58:40.322036 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8128394-5ce0-45e8-98a0-fe9f97d4cc52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:58:40.323462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8128394-5ce0-45e8-98a0-fe9f97d4cc52 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:58:40.334771 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1217/2413] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:58:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:58:42.319988 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f0cec-1371-42c8-833a-73b15248a2d4 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:58:42.324398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f0cec-1371-42c8-833a-73b15248a2d4 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:58:42.324398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f0cec-1371-42c8-833a-73b15248a2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:58:42.340044 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-820f0cec-1371-42c8-833a-73b15248a2d4 None None] MFA Rules not processed for user `0cc36def73aa4bedb9e79d4cf11f5e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:58:42.341408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-820f0cec-1371-42c8-833a-73b15248a2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:58:42.397055 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1197/2414] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:58:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:58:50.329777 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=6 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:58:52.389842 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=79 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:58:59.353390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8128394-5ce0-45e8-98a0-fe9f97d4cc52 service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:58:59.354792 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-c8128394-5ce0-45e8-98a0-fe9f97d4cc52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:58:59.410725 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8128394-5ce0-45e8-98a0-fe9f97d4cc52 service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:58:59.414632 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083a890-fd51-4250-a8d2-7d2594daff9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:58:59.415240 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083a890-fd51-4250-a8d2-7d2594daff9e service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:58:59.415551 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083a890-fd51-4250-a8d2-7d2594daff9e service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:58:59.415826 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083a890-fd51-4250-a8d2-7d2594daff9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:58:59.417563 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-7083a890-fd51-4250-a8d2-7d2594daff9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:58:59.476824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7083a890-fd51-4250-a8d2-7d2594daff9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:58:59.478256 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7083a890-fd51-4250-a8d2-7d2594daff9e service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:58:59.487915 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1218/2415] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:58:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:59:08.687193 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ae838-fb2d-4397-90d8-4261762a6d2b None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:59:08.690443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ae838-fb2d-4397-90d8-4261762a6d2b None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:59:08.690443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ae838-fb2d-4397-90d8-4261762a6d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:59:08.707482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-f58ae838-fb2d-4397-90d8-4261762a6d2b None None] MFA Rules not processed for user `0175a2b440704d51aa14607a2e20e36a`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:59:08.708724 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-f58ae838-fb2d-4397-90d8-4261762a6d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:59:08.715623 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1198/2416] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:59:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:59:09.026150 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083a890-fd51-4250-a8d2-7d2594daff9e service cinder] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:59:09.035627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083a890-fd51-4250-a8d2-7d2594daff9e service cinder] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:59:09.039846 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355424f4-8cb5-42ff-abe0-98d5b4acce6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:59:09.040448 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355424f4-8cb5-42ff-abe0-98d5b4acce6f service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:59:09.040760 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355424f4-8cb5-42ff-abe0-98d5b4acce6f service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:59:09.041034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355424f4-8cb5-42ff-abe0-98d5b4acce6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:59:09.042843 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-355424f4-8cb5-42ff-abe0-98d5b4acce6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:59:09.053091 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355424f4-8cb5-42ff-abe0-98d5b4acce6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:59:09.054597 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355424f4-8cb5-42ff-abe0-98d5b4acce6f service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:59:09.063041 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1219/2417] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:59:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:59:09.447975 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ae838-fb2d-4397-90d8-4261762a6d2b None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:59:09.449491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-f58ae838-fb2d-4397-90d8-4261762a6d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:59:09.520064 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ae838-fb2d-4397-90d8-4261762a6d2b None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:59:09.524852 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1429d5c0-9f6b-4d32-a6d5-4ac2719a4b51 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2876f4182f0949039b862f053533fe85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:59:09.527873 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1429d5c0-9f6b-4d32-a6d5-4ac2719a4b51 service placement] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:59:09.527873 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1429d5c0-9f6b-4d32-a6d5-4ac2719a4b51 service placement] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:59:09.527873 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1429d5c0-9f6b-4d32-a6d5-4ac2719a4b51 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:59:09.527873 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-1429d5c0-9f6b-4d32-a6d5-4ac2719a4b51 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:59:09.540055 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1429d5c0-9f6b-4d32-a6d5-4ac2719a4b51 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:59:09.541662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1429d5c0-9f6b-4d32-a6d5-4ac2719a4b51 service placement] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:59:09.551363 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1199/2418] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:59:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:59:12.609452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355424f4-8cb5-42ff-abe0-98d5b4acce6f service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:59:12.618158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355424f4-8cb5-42ff-abe0-98d5b4acce6f service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:59:12.621759 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6db04-67e2-4f74-83b1-19c571bcf78a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:59:12.622353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6db04-67e2-4f74-83b1-19c571bcf78a service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:59:12.622633 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6db04-67e2-4f74-83b1-19c571bcf78a service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:59:12.622905 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6db04-67e2-4f74-83b1-19c571bcf78a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:59:12.624737 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-7bb6db04-67e2-4f74-83b1-19c571bcf78a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:59:12.634747 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6db04-67e2-4f74-83b1-19c571bcf78a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:59:12.636184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6db04-67e2-4f74-83b1-19c571bcf78a service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:59:12.644529 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1220/2419] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:59:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:59:13.400841 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1429d5c0-9f6b-4d32-a6d5-4ac2719a4b51 service placement] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:59:13.402233 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-1429d5c0-9f6b-4d32-a6d5-4ac2719a4b51 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:59:13.419292 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1429d5c0-9f6b-4d32-a6d5-4ac2719a4b51 service placement] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:59:13.425789 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82057c4b-e48f-4974-a8ab-47f2cb84ee20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:59:13.426397 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82057c4b-e48f-4974-a8ab-47f2cb84ee20 service nova] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:59:13.426678 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82057c4b-e48f-4974-a8ab-47f2cb84ee20 service nova] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:59:13.426952 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82057c4b-e48f-4974-a8ab-47f2cb84ee20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:59:13.428797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-82057c4b-e48f-4974-a8ab-47f2cb84ee20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:59:13.444513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82057c4b-e48f-4974-a8ab-47f2cb84ee20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:59:13.446196 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82057c4b-e48f-4974-a8ab-47f2cb84ee20 service nova] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:59:13.458938 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1200/2420] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:59:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:59:18.839451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6db04-67e2-4f74-83b1-19c571bcf78a service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:59:18.840978 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-7bb6db04-67e2-4f74-83b1-19c571bcf78a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:59:18.856986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6db04-67e2-4f74-83b1-19c571bcf78a service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:59:18.862578 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f285c277-ac59-4fab-bc58-c692d5d2bbf3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:59:18.863174 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285c277-ac59-4fab-bc58-c692d5d2bbf3 service nova] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:59:18.863459 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285c277-ac59-4fab-bc58-c692d5d2bbf3 service nova] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:59:18.863731 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285c277-ac59-4fab-bc58-c692d5d2bbf3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:59:18.865505 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f285c277-ac59-4fab-bc58-c692d5d2bbf3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:59:18.875906 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f285c277-ac59-4fab-bc58-c692d5d2bbf3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:59:18.878102 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f285c277-ac59-4fab-bc58-c692d5d2bbf3 service nova] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:59:18.889961 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1221/2421] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:59:18 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:59:22.565217 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82057c4b-e48f-4974-a8ab-47f2cb84ee20 service nova] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:59:22.576251 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82057c4b-e48f-4974-a8ab-47f2cb84ee20 service nova] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:59:22.582541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64947c-cc27-4d3d-a69e-1780d3d0b199 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:59:22.583243 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64947c-cc27-4d3d-a69e-1780d3d0b199 service nova] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:59:22.583441 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64947c-cc27-4d3d-a69e-1780d3d0b199 service nova] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:59:22.583822 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64947c-cc27-4d3d-a69e-1780d3d0b199 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:59:22.585605 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-6c64947c-cc27-4d3d-a69e-1780d3d0b199 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:59:22.599631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c64947c-cc27-4d3d-a69e-1780d3d0b199 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:59:22.601247 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c64947c-cc27-4d3d-a69e-1780d3d0b199 service nova] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:59:22.611721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1201/2422] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:59:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:59:25.313832 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f285c277-ac59-4fab-bc58-c692d5d2bbf3 service nova] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:59:25.315948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f285c277-ac59-4fab-bc58-c692d5d2bbf3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:59:25.322539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=67 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:59:25.328495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f285c277-ac59-4fab-bc58-c692d5d2bbf3 service nova] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:59:25.332152 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228912b2-b4df-44c1-8036-47dc1b0622c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:59:25.332780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228912b2-b4df-44c1-8036-47dc1b0622c6 service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:59:25.335862 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228912b2-b4df-44c1-8036-47dc1b0622c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:59:25.335862 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228912b2-b4df-44c1-8036-47dc1b0622c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:59:25.335862 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-228912b2-b4df-44c1-8036-47dc1b0622c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:59:25.346655 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228912b2-b4df-44c1-8036-47dc1b0622c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:59:25.348091 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228912b2-b4df-44c1-8036-47dc1b0622c6 service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:59:25.357271 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1222/2423] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:59:25 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:59:32.607132 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=47 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:59:35.363856 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=5 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:59:37.060525 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6fcbb-9da6-4b51-a9fc-b45d00a89e5d None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:59:37.061434 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6fcbb-9da6-4b51-a9fc-b45d00a89e5d None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:59:37.061907 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6fcbb-9da6-4b51-a9fc-b45d00a89e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:59:37.085262 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-8be6fcbb-9da6-4b51-a9fc-b45d00a89e5d None None] MFA Rules not processed for user `bbd9bbb083ab4f5e84e135ec5103cdae`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:59:37.089858 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-8be6fcbb-9da6-4b51-a9fc-b45d00a89e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:59:37.150520 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1202/2424] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Aug 26 00:59:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 00:59:37.265049 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228912b2-b4df-44c1-8036-47dc1b0622c6 service cinder] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:59:37.266684 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-228912b2-b4df-44c1-8036-47dc1b0622c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:59:37.337815 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228912b2-b4df-44c1-8036-47dc1b0622c6 service cinder] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:59:37.342787 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7efe3d1-93cd-46f0-98b1-095e9aa12c07 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f700502671446c797a566f07398a9fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:59:37.343391 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efe3d1-93cd-46f0-98b1-095e9aa12c07 service swift] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:59:37.343665 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efe3d1-93cd-46f0-98b1-095e9aa12c07 service swift] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:59:37.343934 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efe3d1-93cd-46f0-98b1-095e9aa12c07 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:59:37.352550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7efe3d1-93cd-46f0-98b1-095e9aa12c07 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:59:37.354366 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7efe3d1-93cd-46f0-98b1-095e9aa12c07 service swift] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:59:37.362904 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1223/2425] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Aug 26 00:59:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 00:59:38.249457 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f9c41-4da2-420c-bad1-639e3a1857cf None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:59:38.250454 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f9c41-4da2-420c-bad1-639e3a1857cf None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:59:38.250975 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f9c41-4da2-420c-bad1-639e3a1857cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:59:38.275953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-744f9c41-4da2-420c-bad1-639e3a1857cf None None] MFA Rules not processed for user `cd45f1813efc4926a30454aaa108ac47`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:59:38.278050 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-744f9c41-4da2-420c-bad1-639e3a1857cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:59:38.357353 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1203/2426] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 00:59:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:59:38.479830 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7efe3d1-93cd-46f0-98b1-095e9aa12c07 service swift] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:59:38.498789 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7efe3d1-93cd-46f0-98b1-095e9aa12c07 service swift] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:59:38.512409 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2895c-df88-43ba-bc86-e9d676571703 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:59:38.514101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2895c-df88-43ba-bc86-e9d676571703 service nova] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:59:38.514596 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2895c-df88-43ba-bc86-e9d676571703 service nova] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:59:38.514874 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2895c-df88-43ba-bc86-e9d676571703 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:59:38.516975 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-b4e2895c-df88-43ba-bc86-e9d676571703 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:59:38.534528 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2895c-df88-43ba-bc86-e9d676571703 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:59:38.536701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2895c-df88-43ba-bc86-e9d676571703 service nova] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:59:38.551678 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1224/2427] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:59:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:59:48.344159 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=50 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:59:48.546220 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=38 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 00:59:55.675721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f9c41-4da2-420c-bad1-639e3a1857cf None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:59:55.686755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f9c41-4da2-420c-bad1-639e3a1857cf None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:59:55.690865 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e616a1b-0b42-4ca4-ac2a-c6125b51477e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:59:55.691470 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e616a1b-0b42-4ca4-ac2a-c6125b51477e service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:59:55.691751 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e616a1b-0b42-4ca4-ac2a-c6125b51477e service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:59:55.692026 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e616a1b-0b42-4ca4-ac2a-c6125b51477e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:59:55.693767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-8e616a1b-0b42-4ca4-ac2a-c6125b51477e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:59:55.763100 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e616a1b-0b42-4ca4-ac2a-c6125b51477e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:59:55.764559 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e616a1b-0b42-4ca4-ac2a-c6125b51477e service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:59:55.773953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1204/2428] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:59:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:59:56.458643 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4c15d-733d-4acf-b056-8a0bfc7b76c5 None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:59:56.461334 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4c15d-733d-4acf-b056-8a0bfc7b76c5 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:59:56.461334 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4c15d-733d-4acf-b056-8a0bfc7b76c5 None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:59:56.461334 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1225/2429] 10.222.0.59 () {54 vars in 921 bytes} [Fri Aug 26 00:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:59:56.466674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3169c-d250-47d3-9896-f8110812e1cd None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:59:56.467155 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3169c-d250-47d3-9896-f8110812e1cd None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:59:56.467530 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3169c-d250-47d3-9896-f8110812e1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:59:56.488783 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-6bd3169c-d250-47d3-9896-f8110812e1cd None None] MFA Rules not processed for user `0602593263ca4b05a657d5b7697f932f`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:59:56.490083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-6bd3169c-d250-47d3-9896-f8110812e1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:59:56.501762 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1205/2430] 10.222.0.59 () {58 vars in 1018 bytes} [Fri Aug 26 00:59:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:59:56.513781 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4c15d-733d-4acf-b056-8a0bfc7b76c5 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:59:56.518627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-02b4c15d-733d-4acf-b056-8a0bfc7b76c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:59:56.586689 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4c15d-733d-4acf-b056-8a0bfc7b76c5 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:59:56.590415 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c638b3-1654-417c-9c5d-8ae2120f396c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abed7ebb0aae4e4693ce08e51a149312', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:59:56.591133 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c638b3-1654-417c-9c5d-8ae2120f396c service glance] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:59:56.591524 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c638b3-1654-417c-9c5d-8ae2120f396c service glance] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:59:56.591891 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c638b3-1654-417c-9c5d-8ae2120f396c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:59:56.603872 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c638b3-1654-417c-9c5d-8ae2120f396c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:59:56.605363 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c638b3-1654-417c-9c5d-8ae2120f396c service glance] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:59:56.617329 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1226/2431] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:59:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:59:56.920402 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69715ed3-b6ad-4284-8c79-dd68b91ee8c6 None None] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:59:56.923190 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69715ed3-b6ad-4284-8c79-dd68b91ee8c6 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:59:56.923734 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69715ed3-b6ad-4284-8c79-dd68b91ee8c6 None None] PATH_INFO: `/` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:59:56.926029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1206/2432] 10.222.0.59 () {54 vars in 921 bytes} [Fri Aug 26 00:59:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 00:59:56.933846 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7bcc3-91e0-418d-97bf-1e3033bd8542 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:59:56.934134 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7bcc3-91e0-418d-97bf-1e3033bd8542 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:59:56.934590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7bcc3-91e0-418d-97bf-1e3033bd8542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:59:56.966630 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-f2c7bcc3-91e0-418d-97bf-1e3033bd8542 None None] MFA Rules not processed for user `4698a49ed4a04f58a488359a2742e5da`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:59:56.968680 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f2c7bcc3-91e0-418d-97bf-1e3033bd8542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:59:56.977156 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1227/2433] 10.222.0.59 () {58 vars in 1018 bytes} [Fri Aug 26 00:59:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:59:56.988913 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69715ed3-b6ad-4284-8c79-dd68b91ee8c6 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:59:57.001023 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69715ed3-b6ad-4284-8c79-dd68b91ee8c6 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:59:57.005470 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4efe69c-f7d8-4a06-a002-71f83f16f573 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:59:57.006274 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4efe69c-f7d8-4a06-a002-71f83f16f573 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:59:57.006707 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4efe69c-f7d8-4a06-a002-71f83f16f573 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:59:57.007158 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4efe69c-f7d8-4a06-a002-71f83f16f573 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:59:57.016301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4efe69c-f7d8-4a06-a002-71f83f16f573 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:59:57.018610 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4efe69c-f7d8-4a06-a002-71f83f16f573 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:59:57.030094 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1207/2434] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:59:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:59:57.122156 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6916a-51b8-4f6b-9084-bc063b12d1dc None None] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:59:57.122443 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6916a-51b8-4f6b-9084-bc063b12d1dc None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:59:57.122715 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6916a-51b8-4f6b-9084-bc063b12d1dc None None] PATH_INFO: `/` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:59:57.124064 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1228/2435] 10.222.0.59 () {54 vars in 921 bytes} [Fri Aug 26 00:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 00:59:57.129960 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8677e-eba7-4235-8d44-dd295a1b2f0f None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:59:57.130318 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8677e-eba7-4235-8d44-dd295a1b2f0f None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:59:57.130664 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8677e-eba7-4235-8d44-dd295a1b2f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:59:57.151000 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-e7e8677e-eba7-4235-8d44-dd295a1b2f0f None None] MFA Rules not processed for user `0602593263ca4b05a657d5b7697f932f`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 00:59:57.152321 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-e7e8677e-eba7-4235-8d44-dd295a1b2f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:59:57.162108 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1208/2436] 10.222.0.59 () {58 vars in 1018 bytes} [Fri Aug 26 00:59:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 00:59:57.173111 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab6916a-51b8-4f6b-9084-bc063b12d1dc None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:59:57.183809 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab6916a-51b8-4f6b-9084-bc063b12d1dc None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:59:57.187389 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef49aad-770e-4399-b85e-4f6fdf62bb79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:59:57.187983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef49aad-770e-4399-b85e-4f6fdf62bb79 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:59:57.188258 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef49aad-770e-4399-b85e-4f6fdf62bb79 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:59:57.189093 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef49aad-770e-4399-b85e-4f6fdf62bb79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:59:57.199072 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef49aad-770e-4399-b85e-4f6fdf62bb79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:59:57.200520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef49aad-770e-4399-b85e-4f6fdf62bb79 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:59:57.212656 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1229/2437] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:59:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 00:59:59.896965 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8677e-eba7-4235-8d44-dd295a1b2f0f None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 00:59:59.908308 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8677e-eba7-4235-8d44-dd295a1b2f0f None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 00:59:59.913695 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54728e55-4817-4f06-a905-bbb7424e643c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2876f4182f0949039b862f053533fe85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 00:59:59.914300 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54728e55-4817-4f06-a905-bbb7424e643c service placement] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 00:59:59.914581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54728e55-4817-4f06-a905-bbb7424e643c service placement] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 00:59:59.914983 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54728e55-4817-4f06-a905-bbb7424e643c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 00:59:59.916883 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-54728e55-4817-4f06-a905-bbb7424e643c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 00:59:59.929311 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54728e55-4817-4f06-a905-bbb7424e643c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 00:59:59.932389 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54728e55-4817-4f06-a905-bbb7424e643c service placement] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 00:59:59.943339 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1209/2438] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 00:59:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:00:00.318936 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef49aad-770e-4399-b85e-4f6fdf62bb79 service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:00.320990 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-8ef49aad-770e-4399-b85e-4f6fdf62bb79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:00.335205 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef49aad-770e-4399-b85e-4f6fdf62bb79 service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:00.340183 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6617db3f-758c-42fa-a477-069362fb1084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:00.341648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617db3f-758c-42fa-a477-069362fb1084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:00.342118 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617db3f-758c-42fa-a477-069362fb1084 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:00.342549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617db3f-758c-42fa-a477-069362fb1084 admin admin] PATH_INFO: `/v3/users/0cc36def73aa4bedb9e79d4cf11f5e8c` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:00.344085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6617db3f-758c-42fa-a477-069362fb1084 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cc36def73aa4bedb9e79d4cf11f5e8c)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:00.353331 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6617db3f-758c-42fa-a477-069362fb1084 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:00.542511 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-6617db3f-758c-42fa-a477-069362fb1084 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cc36def73aa4bedb9e79d4cf11f5e8c'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:00.543076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-6617db3f-758c-42fa-a477-069362fb1084 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cc36def73aa4bedb9e79d4cf11f5e8c'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:00.555710 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-6617db3f-758c-42fa-a477-069362fb1084 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cc36def73aa4bedb9e79d4cf11f5e8c'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:00.654536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-6617db3f-758c-42fa-a477-069362fb1084 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cc36def73aa4bedb9e79d4cf11f5e8c'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:00.655171 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-6617db3f-758c-42fa-a477-069362fb1084 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cc36def73aa4bedb9e79d4cf11f5e8c'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:00.668836 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1230/2439] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:00:00 2022] DELETE /identity/v3/users/0cc36def73aa4bedb9e79d4cf11f5e8c => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:01.082220 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54728e55-4817-4f06-a905-bbb7424e643c service placement] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:01.105335 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54728e55-4817-4f06-a905-bbb7424e643c service placement] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:01.171870 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fdc7b4-e620-4295-840d-0903b6d51f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:01.173379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fdc7b4-e620-4295-840d-0903b6d51f4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:01.173652 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fdc7b4-e620-4295-840d-0903b6d51f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:01.173918 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fdc7b4-e620-4295-840d-0903b6d51f4e admin admin] PATH_INFO: `/v3/projects/368f6a9e030e4aff9ef6559a6a165975` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:01.175035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fdc7b4-e620-4295-840d-0903b6d51f4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=368f6a9e030e4aff9ef6559a6a165975)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:01.176956 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fdc7b4-e620-4295-840d-0903b6d51f4e admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:01.288998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-91fdc7b4-e620-4295-840d-0903b6d51f4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '368f6a9e030e4aff9ef6559a6a165975', 'request_id': 'req-91fdc7b4-e620-4295-840d-0903b6d51f4e'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:01.351677 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-91fdc7b4-e620-4295-840d-0903b6d51f4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '368f6a9e030e4aff9ef6559a6a165975', 'request_id': 'req-91fdc7b4-e620-4295-840d-0903b6d51f4e'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:01.354120 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:01.359969 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-91fdc7b4-e620-4295-840d-0903b6d51f4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '368f6a9e030e4aff9ef6559a6a165975', 'request_id': 'req-91fdc7b4-e620-4295-840d-0903b6d51f4e'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:01.373318 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-91fdc7b4-e620-4295-840d-0903b6d51f4e admin admin] The token cache is being invalidate because project 368f6a9e030e4aff9ef6559a6a165975 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:00:01.373648 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-91fdc7b4-e620-4295-840d-0903b6d51f4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:01.374201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-91fdc7b4-e620-4295-840d-0903b6d51f4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:01.379066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1210/2440] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:00:01 2022] DELETE /identity/v3/projects/368f6a9e030e4aff9ef6559a6a165975 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:01.381739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d587bf-c7ef-45c1-bf17-004d8ae18ad3 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:01.382572 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d587bf-c7ef-45c1-bf17-004d8ae18ad3 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:01.383466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d587bf-c7ef-45c1-bf17-004d8ae18ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:01.406361 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698772c4-0b36-4e01-b208-ae07ff24e18b None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:01.407915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698772c4-0b36-4e01-b208-ae07ff24e18b None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:01.408227 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698772c4-0b36-4e01-b208-ae07ff24e18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:01.415465 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-16d587bf-c7ef-45c1-bf17-004d8ae18ad3 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:01.416822 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-16d587bf-c7ef-45c1-bf17-004d8ae18ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:01.432259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-698772c4-0b36-4e01-b208-ae07ff24e18b None None] MFA Rules not processed for user `bbd9bbb083ab4f5e84e135ec5103cdae`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:01.433585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-698772c4-0b36-4e01-b208-ae07ff24e18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:01.471818 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:01.487143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1231/2441] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:00:01.494142 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9707133-2bbb-45c7-bab7-2aed4bdbe2c2 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:01.494432 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9707133-2bbb-45c7-bab7-2aed4bdbe2c2 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:01.495926 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9707133-2bbb-45c7-bab7-2aed4bdbe2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:01.523203 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1211/2442] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Aug 26 01:00:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:00:01.530641 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-a9707133-2bbb-45c7-bab7-2aed4bdbe2c2 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:01.532267 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a9707133-2bbb-45c7-bab7-2aed4bdbe2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:01.542363 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698772c4-0b36-4e01-b208-ae07ff24e18b None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:01.547202 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-698772c4-0b36-4e01-b208-ae07ff24e18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:01.552279 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1232/2443] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:00:01.555593 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9707133-2bbb-45c7-bab7-2aed4bdbe2c2 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:01.575115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9707133-2bbb-45c7-bab7-2aed4bdbe2c2 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:01.585371 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9baf229-d4ef-4ca4-87d0-87dda58156f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:01.586123 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9baf229-d4ef-4ca4-87d0-87dda58156f8 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:01.586514 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9baf229-d4ef-4ca4-87d0-87dda58156f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:01.586857 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9baf229-d4ef-4ca4-87d0-87dda58156f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:01.587659 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9baf229-d4ef-4ca4-87d0-87dda58156f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 01:00:01.588037 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9baf229-d4ef-4ca4-87d0-87dda58156f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:01.590056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9baf229-d4ef-4ca4-87d0-87dda58156f8 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:01.611286 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1233/2444] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 01:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 01:00:01.615753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9baf229-d4ef-4ca4-87d0-87dda58156f8 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:01.627002 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9baf229-d4ef-4ca4-87d0-87dda58156f8 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:01.635219 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283526dc-6b66-4bb0-a968-6a1bee584dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:01.636155 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283526dc-6b66-4bb0-a968-6a1bee584dce admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:01.636694 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283526dc-6b66-4bb0-a968-6a1bee584dce admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:01.637266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283526dc-6b66-4bb0-a968-6a1bee584dce admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:01.638138 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283526dc-6b66-4bb0-a968-6a1bee584dce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:01.640433 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283526dc-6b66-4bb0-a968-6a1bee584dce admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:01.665613 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698772c4-0b36-4e01-b208-ae07ff24e18b None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:01.671024 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c53c92-0b72-4a4e-8cf9-1ad9fe08a52a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f700502671446c797a566f07398a9fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:01.671638 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c53c92-0b72-4a4e-8cf9-1ad9fe08a52a service swift] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:01.671915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c53c92-0b72-4a4e-8cf9-1ad9fe08a52a service swift] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:01.672258 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c53c92-0b72-4a4e-8cf9-1ad9fe08a52a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:01.682281 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c53c92-0b72-4a4e-8cf9-1ad9fe08a52a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:01.683877 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c53c92-0b72-4a4e-8cf9-1ad9fe08a52a service swift] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:01.695264 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1212/2445] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Aug 26 01:00:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 01:00:01.703091 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1234/2446] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:00:01 2022] POST /identity/v3/projects => generated 385 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 01:00:01.708095 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c53c92-0b72-4a4e-8cf9-1ad9fe08a52a service swift] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:01.720991 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c53c92-0b72-4a4e-8cf9-1ad9fe08a52a service swift] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:01.791564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14643a9-0319-4eab-a2b9-07b096ce1de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:01.792188 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14643a9-0319-4eab-a2b9-07b096ce1de2 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:01.792466 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14643a9-0319-4eab-a2b9-07b096ce1de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:01.792764 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14643a9-0319-4eab-a2b9-07b096ce1de2 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:01.793382 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14643a9-0319-4eab-a2b9-07b096ce1de2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:01.795345 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14643a9-0319-4eab-a2b9-07b096ce1de2 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:01.906111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1213/2447] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:00:01 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 01:00:01.908108 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283526dc-6b66-4bb0-a968-6a1bee584dce admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:01.925519 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283526dc-6b66-4bb0-a968-6a1bee584dce admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:01.934945 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd2848-2956-46db-8776-c3898b92237a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:01.935657 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd2848-2956-46db-8776-c3898b92237a admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:01.936051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd2848-2956-46db-8776-c3898b92237a admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:01.937871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd2848-2956-46db-8776-c3898b92237a admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:01.937871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd2848-2956-46db-8776-c3898b92237a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:01.939691 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd2848-2956-46db-8776-c3898b92237a admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:01.957397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1235/2448] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:00:01.957647 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14643a9-0319-4eab-a2b9-07b096ce1de2 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:01.976603 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14643a9-0319-4eab-a2b9-07b096ce1de2 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:01.993720 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a835b122-e5ec-4576-9186-24cbf55c98d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:01.994352 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835b122-e5ec-4576-9186-24cbf55c98d9 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:01.994628 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835b122-e5ec-4576-9186-24cbf55c98d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:01.994913 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835b122-e5ec-4576-9186-24cbf55c98d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:01.995555 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a835b122-e5ec-4576-9186-24cbf55c98d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:01.997253 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a835b122-e5ec-4576-9186-24cbf55c98d9 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:01.999796 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:02.016527 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1214/2449] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 01:00:02.020374 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd2848-2956-46db-8776-c3898b92237a admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:02.034916 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd2848-2956-46db-8776-c3898b92237a admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:02.052629 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d0708-16af-44f0-8353-98a8e16c9cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:02.053740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d0708-16af-44f0-8353-98a8e16c9cad admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:02.054206 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d0708-16af-44f0-8353-98a8e16c9cad admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:02.055254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d0708-16af-44f0-8353-98a8e16c9cad admin admin] PATH_INFO: `/v3/projects/1173abf0d8774c4090d36ae3546c961b/users/6853ce2599fa4099940e6a5791b756d1/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:02.078990 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181d0708-16af-44f0-8353-98a8e16c9cad admin admin] RBAC: Authorizing `identity:create_grant(project_id=1173abf0d8774c4090d36ae3546c961b, user_id=6853ce2599fa4099940e6a5791b756d1, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:02.083757 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181d0708-16af-44f0-8353-98a8e16c9cad admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:02.198383 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1236/2450] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:00:02 2022] PUT /identity/v3/projects/1173abf0d8774c4090d36ae3546c961b/users/6853ce2599fa4099940e6a5791b756d1/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:02.210282 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a887e-a22e-4392-9e75-cfd549416764 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:02.210824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a887e-a22e-4392-9e75-cfd549416764 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:02.210824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a887e-a22e-4392-9e75-cfd549416764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:02.264603 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-ff5a887e-a22e-4392-9e75-cfd549416764 None None] MFA Rules not processed for user `6853ce2599fa4099940e6a5791b756d1`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:02.266118 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-ff5a887e-a22e-4392-9e75-cfd549416764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:02.372703 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1215/2451] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:00:02.382364 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d0708-16af-44f0-8353-98a8e16c9cad admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:02.392424 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-181d0708-16af-44f0-8353-98a8e16c9cad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:02.489389 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d0708-16af-44f0-8353-98a8e16c9cad admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:02.496795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc528949-105a-46b6-85ee-e01807d9b374 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:02.497555 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc528949-105a-46b6-85ee-e01807d9b374 service nova] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:02.497945 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc528949-105a-46b6-85ee-e01807d9b374 service nova] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:02.498328 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc528949-105a-46b6-85ee-e01807d9b374 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:02.507241 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc528949-105a-46b6-85ee-e01807d9b374 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:02.509976 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc528949-105a-46b6-85ee-e01807d9b374 service nova] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:02.523270 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1237/2452] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:00:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:00:12.203712 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a887e-a22e-4392-9e75-cfd549416764 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:12.263506 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a887e-a22e-4392-9e75-cfd549416764 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:12.267909 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd9eba-9a53-4325-88e7-a3a1cdbf0a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:12.268559 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd9eba-9a53-4325-88e7-a3a1cdbf0a75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:12.268843 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd9eba-9a53-4325-88e7-a3a1cdbf0a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:12.269118 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd9eba-9a53-4325-88e7-a3a1cdbf0a75 admin admin] PATH_INFO: `/v3/users/6853ce2599fa4099940e6a5791b756d1` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:12.270228 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd9eba-9a53-4325-88e7-a3a1cdbf0a75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6853ce2599fa4099940e6a5791b756d1)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:12.272140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd9eba-9a53-4325-88e7-a3a1cdbf0a75 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:12.466247 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-40fd9eba-9a53-4325-88e7-a3a1cdbf0a75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6853ce2599fa4099940e6a5791b756d1'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:12.479586 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-40fd9eba-9a53-4325-88e7-a3a1cdbf0a75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6853ce2599fa4099940e6a5791b756d1'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:12.516384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=69,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:12.522936 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-40fd9eba-9a53-4325-88e7-a3a1cdbf0a75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6853ce2599fa4099940e6a5791b756d1'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:12.523562 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-40fd9eba-9a53-4325-88e7-a3a1cdbf0a75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6853ce2599fa4099940e6a5791b756d1'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:12.524178 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-40fd9eba-9a53-4325-88e7-a3a1cdbf0a75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6853ce2599fa4099940e6a5791b756d1'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:12.538425 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1216/2453] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:00:12 2022] DELETE /identity/v3/users/6853ce2599fa4099940e6a5791b756d1 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:12.543573 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59512f-4803-49de-9efc-53126857cec7 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:12.544061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59512f-4803-49de-9efc-53126857cec7 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:12.544350 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59512f-4803-49de-9efc-53126857cec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:12.568713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-4a59512f-4803-49de-9efc-53126857cec7 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:12.570277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-4a59512f-4803-49de-9efc-53126857cec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:12.636090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1238/2454] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 01:00:12.647195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd9eba-9a53-4325-88e7-a3a1cdbf0a75 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:12.648659 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-40fd9eba-9a53-4325-88e7-a3a1cdbf0a75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:12.659249 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:12.707065 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd9eba-9a53-4325-88e7-a3a1cdbf0a75 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:12.710816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6972f8-8726-40e3-888d-cf1f4f4383a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:12.711425 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6972f8-8726-40e3-888d-cf1f4f4383a6 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:12.711869 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6972f8-8726-40e3-888d-cf1f4f4383a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:12.712531 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6972f8-8726-40e3-888d-cf1f4f4383a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:12.722054 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6972f8-8726-40e3-888d-cf1f4f4383a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:12.727139 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6972f8-8726-40e3-888d-cf1f4f4383a6 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:12.732913 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1217/2455] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:00:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:00:13.101420 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59512f-4803-49de-9efc-53126857cec7 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:13.102856 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-4a59512f-4803-49de-9efc-53126857cec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:13.115374 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59512f-4803-49de-9efc-53126857cec7 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:13.120154 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198ca25-f141-4282-8814-66bec34c3351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:13.120929 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198ca25-f141-4282-8814-66bec34c3351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:13.121284 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198ca25-f141-4282-8814-66bec34c3351 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:13.121629 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198ca25-f141-4282-8814-66bec34c3351 admin admin] PATH_INFO: `/v3/projects/1173abf0d8774c4090d36ae3546c961b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:13.122851 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b198ca25-f141-4282-8814-66bec34c3351 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1173abf0d8774c4090d36ae3546c961b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:13.124867 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b198ca25-f141-4282-8814-66bec34c3351 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:13.237517 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-b198ca25-f141-4282-8814-66bec34c3351 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1173abf0d8774c4090d36ae3546c961b', 'request_id': 'req-b198ca25-f141-4282-8814-66bec34c3351'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:13.311353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-b198ca25-f141-4282-8814-66bec34c3351 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1173abf0d8774c4090d36ae3546c961b', 'request_id': 'req-b198ca25-f141-4282-8814-66bec34c3351'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:13.318321 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-b198ca25-f141-4282-8814-66bec34c3351 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1173abf0d8774c4090d36ae3546c961b', 'request_id': 'req-b198ca25-f141-4282-8814-66bec34c3351'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:13.330544 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-b198ca25-f141-4282-8814-66bec34c3351 admin admin] The token cache is being invalidate because project 1173abf0d8774c4090d36ae3546c961b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:00:13.330831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-b198ca25-f141-4282-8814-66bec34c3351 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:13.331479 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-b198ca25-f141-4282-8814-66bec34c3351 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:13.333009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1239/2456] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:00:13 2022] DELETE /identity/v3/projects/1173abf0d8774c4090d36ae3546c961b => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:13.337941 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141ded7-da6e-4d41-8e06-edd7aacfaa40 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:13.338426 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141ded7-da6e-4d41-8e06-edd7aacfaa40 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:13.338548 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141ded7-da6e-4d41-8e06-edd7aacfaa40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:13.358011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-c141ded7-da6e-4d41-8e06-edd7aacfaa40 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:13.359216 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-c141ded7-da6e-4d41-8e06-edd7aacfaa40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:13.417731 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1218/2457] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:00:13.425592 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23807c4c-2120-40d8-aa14-aa435bd2424d None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:13.425884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23807c4c-2120-40d8-aa14-aa435bd2424d None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:13.426164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23807c4c-2120-40d8-aa14-aa435bd2424d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:13.449187 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-23807c4c-2120-40d8-aa14-aa435bd2424d None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:13.450390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-23807c4c-2120-40d8-aa14-aa435bd2424d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:13.459217 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1240/2458] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:00:13.463390 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c141ded7-da6e-4d41-8e06-edd7aacfaa40 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:13.473254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c141ded7-da6e-4d41-8e06-edd7aacfaa40 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:13.477799 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b26c6-73ea-4dd9-96c0-2b818bac4cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:13.478389 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b26c6-73ea-4dd9-96c0-2b818bac4cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:13.478663 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b26c6-73ea-4dd9-96c0-2b818bac4cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:13.478927 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b26c6-73ea-4dd9-96c0-2b818bac4cb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:13.479581 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b26c6-73ea-4dd9-96c0-2b818bac4cb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 01:00:13.479879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b26c6-73ea-4dd9-96c0-2b818bac4cb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:13.481485 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b26c6-73ea-4dd9-96c0-2b818bac4cb7 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:13.492793 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1219/2459] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 01:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 01:00:13.497066 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23807c4c-2120-40d8-aa14-aa435bd2424d None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:13.506673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23807c4c-2120-40d8-aa14-aa435bd2424d None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:13.511282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55f091-6868-494b-adf1-969a81def713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:13.511877 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55f091-6868-494b-adf1-969a81def713 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:13.512158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55f091-6868-494b-adf1-969a81def713 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:13.512434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55f091-6868-494b-adf1-969a81def713 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:13.513089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f55f091-6868-494b-adf1-969a81def713 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:13.515032 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f55f091-6868-494b-adf1-969a81def713 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:13.656027 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1241/2460] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:00:13 2022] POST /identity/v3/projects => generated 403 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:00:13.660023 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b26c6-73ea-4dd9-96c0-2b818bac4cb7 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:13.670427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b26c6-73ea-4dd9-96c0-2b818bac4cb7 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:13.728311 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b0219-5d1c-4cbc-a9cc-69d6f4d2a18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:13.728956 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b0219-5d1c-4cbc-a9cc-69d6f4d2a18c admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:13.729241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b0219-5d1c-4cbc-a9cc-69d6f4d2a18c admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:13.729517 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b0219-5d1c-4cbc-a9cc-69d6f4d2a18c admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:13.730141 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857b0219-5d1c-4cbc-a9cc-69d6f4d2a18c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:13.732052 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857b0219-5d1c-4cbc-a9cc-69d6f4d2a18c admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:13.815505 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1220/2461] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:00:13 2022] POST /identity/v3/users => generated 302 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:00:13.819719 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55f091-6868-494b-adf1-969a81def713 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:13.830542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55f091-6868-494b-adf1-969a81def713 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:13.835314 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4b6ce-6bda-4adb-8fe1-7d002b3643bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:13.835934 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4b6ce-6bda-4adb-8fe1-7d002b3643bd admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:13.836215 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4b6ce-6bda-4adb-8fe1-7d002b3643bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:13.836513 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4b6ce-6bda-4adb-8fe1-7d002b3643bd admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:13.837378 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4b6ce-6bda-4adb-8fe1-7d002b3643bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:13.838987 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4b6ce-6bda-4adb-8fe1-7d002b3643bd admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:13.849093 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1242/2462] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:00:13.853420 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b0219-5d1c-4cbc-a9cc-69d6f4d2a18c admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:13.864306 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b0219-5d1c-4cbc-a9cc-69d6f4d2a18c admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:13.868991 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416b417-73a8-40c5-b239-aad2adb0b5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:13.869619 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416b417-73a8-40c5-b239-aad2adb0b5df admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:13.869897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416b417-73a8-40c5-b239-aad2adb0b5df admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:13.870173 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416b417-73a8-40c5-b239-aad2adb0b5df admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:13.870813 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5416b417-73a8-40c5-b239-aad2adb0b5df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:13.872405 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5416b417-73a8-40c5-b239-aad2adb0b5df admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:13.883498 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1221/2463] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:00:13.887796 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4b6ce-6bda-4adb-8fe1-7d002b3643bd admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:13.897826 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4b6ce-6bda-4adb-8fe1-7d002b3643bd admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:13.903007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b0ee3-fb82-4316-b5e4-df0fa8e217f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:13.903658 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b0ee3-fb82-4316-b5e4-df0fa8e217f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:13.903942 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b0ee3-fb82-4316-b5e4-df0fa8e217f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:13.904218 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b0ee3-fb82-4316-b5e4-df0fa8e217f8 admin admin] PATH_INFO: `/v3/projects/e771e748e1f545b0aec7211e645b8d63/users/8645d358a3e3487a893fa4d29e5015ee/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:13.919120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61b0ee3-fb82-4316-b5e4-df0fa8e217f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e771e748e1f545b0aec7211e645b8d63, user_id=8645d358a3e3487a893fa4d29e5015ee, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:13.921856 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61b0ee3-fb82-4316-b5e4-df0fa8e217f8 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:13.996695 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1243/2464] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:00:13 2022] PUT /identity/v3/projects/e771e748e1f545b0aec7211e645b8d63/users/8645d358a3e3487a893fa4d29e5015ee/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:14.007711 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082dee72-4f5b-440c-89c8-bb5eea4eb688 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:14.007942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082dee72-4f5b-440c-89c8-bb5eea4eb688 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:14.008214 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082dee72-4f5b-440c-89c8-bb5eea4eb688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:14.035879 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-082dee72-4f5b-440c-89c8-bb5eea4eb688 None None] MFA Rules not processed for user `8645d358a3e3487a893fa4d29e5015ee`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:14.037118 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-082dee72-4f5b-440c-89c8-bb5eea4eb688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:14.041223 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:14.100976 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1222/2465] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:14 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:00:14.105551 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b0ee3-fb82-4316-b5e4-df0fa8e217f8 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:14.127600 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:14.168580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b0ee3-fb82-4316-b5e4-df0fa8e217f8 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:14.173638 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef7514-5e9c-41a5-89de-dbb8e240fefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:14.174245 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef7514-5e9c-41a5-89de-dbb8e240fefe admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:14.174529 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef7514-5e9c-41a5-89de-dbb8e240fefe admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:14.174804 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef7514-5e9c-41a5-89de-dbb8e240fefe admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:14.175435 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef7514-5e9c-41a5-89de-dbb8e240fefe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:14.177399 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef7514-5e9c-41a5-89de-dbb8e240fefe admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:14.243834 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1244/2466] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:00:14 2022] POST /identity/v3/projects => generated 405 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:00:14.248031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082dee72-4f5b-440c-89c8-bb5eea4eb688 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:14.259339 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082dee72-4f5b-440c-89c8-bb5eea4eb688 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:14.313866 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd716d0-dc77-41bd-b1a2-76dda9afe6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:14.314633 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd716d0-dc77-41bd-b1a2-76dda9afe6aa admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:14.314904 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd716d0-dc77-41bd-b1a2-76dda9afe6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:14.315171 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd716d0-dc77-41bd-b1a2-76dda9afe6aa admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:14.315792 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd716d0-dc77-41bd-b1a2-76dda9afe6aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:14.317736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd716d0-dc77-41bd-b1a2-76dda9afe6aa admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:14.400664 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1223/2467] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:00:14 2022] POST /identity/v3/users => generated 309 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:00:14.402417 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef7514-5e9c-41a5-89de-dbb8e240fefe admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:14.416926 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef7514-5e9c-41a5-89de-dbb8e240fefe admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:14.421876 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2fb82-1544-4a9a-ac2e-69bc9bbcc62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:14.422491 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2fb82-1544-4a9a-ac2e-69bc9bbcc62e admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:14.422773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2fb82-1544-4a9a-ac2e-69bc9bbcc62e admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:14.423047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2fb82-1544-4a9a-ac2e-69bc9bbcc62e admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:14.423700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa2fb82-1544-4a9a-ac2e-69bc9bbcc62e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:14.425339 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa2fb82-1544-4a9a-ac2e-69bc9bbcc62e admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:14.439798 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1245/2468] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:00:14.441041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd716d0-dc77-41bd-b1a2-76dda9afe6aa admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:14.451259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd716d0-dc77-41bd-b1a2-76dda9afe6aa admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:14.457338 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ce794-f415-4cfc-be38-b66209234f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:14.458005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ce794-f415-4cfc-be38-b66209234f5a admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:14.458287 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ce794-f415-4cfc-be38-b66209234f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:14.458561 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ce794-f415-4cfc-be38-b66209234f5a admin admin] PATH_INFO: `/v3/projects/5a61bcb96f00420aa0ec9d8ec3a1fffa/users/dbdfa897dd3d4070a8cdc22548d76350/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:14.475526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ce794-f415-4cfc-be38-b66209234f5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a61bcb96f00420aa0ec9d8ec3a1fffa, user_id=dbdfa897dd3d4070a8cdc22548d76350, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:14.478403 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ce794-f415-4cfc-be38-b66209234f5a admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:14.557546 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1224/2469] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:00:14 2022] PUT /identity/v3/projects/5a61bcb96f00420aa0ec9d8ec3a1fffa/users/dbdfa897dd3d4070a8cdc22548d76350/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:14.561651 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2fb82-1544-4a9a-ac2e-69bc9bbcc62e admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:14.572196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2fb82-1544-4a9a-ac2e-69bc9bbcc62e admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:14.639591 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b87e0-5d34-41b6-86eb-493916bc9189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:14.640217 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b87e0-5d34-41b6-86eb-493916bc9189 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:14.640521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b87e0-5d34-41b6-86eb-493916bc9189 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:14.640904 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b87e0-5d34-41b6-86eb-493916bc9189 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:14.641449 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689b87e0-5d34-41b6-86eb-493916bc9189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:14.643685 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689b87e0-5d34-41b6-86eb-493916bc9189 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:14.653780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1246/2470] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:00:14.657968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ce794-f415-4cfc-be38-b66209234f5a admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:14.668730 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ce794-f415-4cfc-be38-b66209234f5a admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:14.673517 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde5bca-d49f-46d3-900e-a25e52d6e5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:14.674162 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde5bca-d49f-46d3-900e-a25e52d6e5e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:14.676804 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde5bca-d49f-46d3-900e-a25e52d6e5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:14.676804 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde5bca-d49f-46d3-900e-a25e52d6e5e5 admin admin] PATH_INFO: `/v3/projects/5a61bcb96f00420aa0ec9d8ec3a1fffa/users/dbdfa897dd3d4070a8cdc22548d76350/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:14.676804 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde5bca-d49f-46d3-900e-a25e52d6e5e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a61bcb96f00420aa0ec9d8ec3a1fffa, user_id=dbdfa897dd3d4070a8cdc22548d76350, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:14.679269 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde5bca-d49f-46d3-900e-a25e52d6e5e5 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:14.690635 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1225/2471] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:00:14 2022] PUT /identity/v3/projects/5a61bcb96f00420aa0ec9d8ec3a1fffa/users/dbdfa897dd3d4070a8cdc22548d76350/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 01:00:14.702297 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d7bbf-b194-4445-a969-8d638911cff1 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:14.702833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d7bbf-b194-4445-a969-8d638911cff1 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:14.702833 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d7bbf-b194-4445-a969-8d638911cff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:14.731633 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-c25d7bbf-b194-4445-a969-8d638911cff1 None None] MFA Rules not processed for user `dbdfa897dd3d4070a8cdc22548d76350`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:14.732896 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-c25d7bbf-b194-4445-a969-8d638911cff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:14.806244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1247/2472] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:14 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:00:14.817224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde5bca-d49f-46d3-900e-a25e52d6e5e5 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:14.818978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-bcde5bca-d49f-46d3-900e-a25e52d6e5e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:14.881466 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde5bca-d49f-46d3-900e-a25e52d6e5e5 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:14.885224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e2a4f-9075-43c0-b469-c5b322fc2484 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:14.886466 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e2a4f-9075-43c0-b469-c5b322fc2484 service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:14.886752 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e2a4f-9075-43c0-b469-c5b322fc2484 service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:14.887025 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e2a4f-9075-43c0-b469-c5b322fc2484 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:14.896664 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e2a4f-9075-43c0-b469-c5b322fc2484 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:14.898117 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e2a4f-9075-43c0-b469-c5b322fc2484 service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:14.903701 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:14.909856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1226/2473] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:00:14 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:00:15.063022 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d7bbf-b194-4445-a969-8d638911cff1 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:15.082519 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:15.126929 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d7bbf-b194-4445-a969-8d638911cff1 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:15.132106 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affe9d8-1f95-4181-bc53-e43d779c5935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:15.132769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affe9d8-1f95-4181-bc53-e43d779c5935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:15.133054 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affe9d8-1f95-4181-bc53-e43d779c5935 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:15.133329 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affe9d8-1f95-4181-bc53-e43d779c5935 admin admin] PATH_INFO: `/v3/users/8645d358a3e3487a893fa4d29e5015ee` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:15.134558 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6affe9d8-1f95-4181-bc53-e43d779c5935 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8645d358a3e3487a893fa4d29e5015ee)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:15.136542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6affe9d8-1f95-4181-bc53-e43d779c5935 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:15.312611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-6affe9d8-1f95-4181-bc53-e43d779c5935 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8645d358a3e3487a893fa4d29e5015ee'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:15.313348 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-6affe9d8-1f95-4181-bc53-e43d779c5935 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8645d358a3e3487a893fa4d29e5015ee'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:15.325711 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-6affe9d8-1f95-4181-bc53-e43d779c5935 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8645d358a3e3487a893fa4d29e5015ee'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:15.380081 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-6affe9d8-1f95-4181-bc53-e43d779c5935 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8645d358a3e3487a893fa4d29e5015ee'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:15.381001 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-6affe9d8-1f95-4181-bc53-e43d779c5935 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8645d358a3e3487a893fa4d29e5015ee'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:15.395034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1248/2474] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:00:15 2022] DELETE /identity/v3/users/8645d358a3e3487a893fa4d29e5015ee => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:15.399522 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906320f-5de8-4b86-bffb-1948c542a1dd None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:15.399810 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906320f-5de8-4b86-bffb-1948c542a1dd None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:15.400114 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906320f-5de8-4b86-bffb-1948c542a1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:15.419100 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-4906320f-5de8-4b86-bffb-1948c542a1dd None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:15.420278 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-4906320f-5de8-4b86-bffb-1948c542a1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:15.478380 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1227/2475] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:00:15.488229 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affe9d8-1f95-4181-bc53-e43d779c5935 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:15.489662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-6affe9d8-1f95-4181-bc53-e43d779c5935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:15.547694 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affe9d8-1f95-4181-bc53-e43d779c5935 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:15.552054 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a804171-ba23-4de8-a664-94a9b3bb0e36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:15.552826 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a804171-ba23-4de8-a664-94a9b3bb0e36 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:15.553109 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a804171-ba23-4de8-a664-94a9b3bb0e36 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:15.553382 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a804171-ba23-4de8-a664-94a9b3bb0e36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:15.562445 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a804171-ba23-4de8-a664-94a9b3bb0e36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:15.563930 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a804171-ba23-4de8-a664-94a9b3bb0e36 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:15.573742 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1249/2476] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:00:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:00:15.953435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906320f-5de8-4b86-bffb-1948c542a1dd None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:15.954876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-4906320f-5de8-4b86-bffb-1948c542a1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:15.966785 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906320f-5de8-4b86-bffb-1948c542a1dd None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:15.971823 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f69db-dd66-4680-81bb-3ff3999edfbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:15.972458 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f69db-dd66-4680-81bb-3ff3999edfbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:15.972763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f69db-dd66-4680-81bb-3ff3999edfbf admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:15.973036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f69db-dd66-4680-81bb-3ff3999edfbf admin admin] PATH_INFO: `/v3/projects/e771e748e1f545b0aec7211e645b8d63` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:15.974164 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f69db-dd66-4680-81bb-3ff3999edfbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e771e748e1f545b0aec7211e645b8d63)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:15.976062 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f69db-dd66-4680-81bb-3ff3999edfbf admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:16.065016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c22f69db-dd66-4680-81bb-3ff3999edfbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e771e748e1f545b0aec7211e645b8d63', 'request_id': 'req-c22f69db-dd66-4680-81bb-3ff3999edfbf'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:16.111482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c22f69db-dd66-4680-81bb-3ff3999edfbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e771e748e1f545b0aec7211e645b8d63', 'request_id': 'req-c22f69db-dd66-4680-81bb-3ff3999edfbf'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:16.119252 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c22f69db-dd66-4680-81bb-3ff3999edfbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e771e748e1f545b0aec7211e645b8d63', 'request_id': 'req-c22f69db-dd66-4680-81bb-3ff3999edfbf'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:16.129941 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c22f69db-dd66-4680-81bb-3ff3999edfbf admin admin] The token cache is being invalidate because project e771e748e1f545b0aec7211e645b8d63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:00:16.130479 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c22f69db-dd66-4680-81bb-3ff3999edfbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:16.131127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c22f69db-dd66-4680-81bb-3ff3999edfbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:16.132618 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1228/2477] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:00:15 2022] DELETE /identity/v3/projects/e771e748e1f545b0aec7211e645b8d63 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 01:00:16.136894 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a804171-ba23-4de8-a664-94a9b3bb0e36 service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:16.146807 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a804171-ba23-4de8-a664-94a9b3bb0e36 service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:16.181515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:16.205190 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eed10b-98af-4fad-9a52-53d5d0127ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:16.205861 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eed10b-98af-4fad-9a52-53d5d0127ebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:16.206162 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eed10b-98af-4fad-9a52-53d5d0127ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:16.206509 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eed10b-98af-4fad-9a52-53d5d0127ebf admin admin] PATH_INFO: `/v3/users/dbdfa897dd3d4070a8cdc22548d76350` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:16.208088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eed10b-98af-4fad-9a52-53d5d0127ebf admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbdfa897dd3d4070a8cdc22548d76350)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:16.210521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eed10b-98af-4fad-9a52-53d5d0127ebf admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:16.306253 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bbf22-640e-4598-bfd8-9170af3cb8f3 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:16.306542 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bbf22-640e-4598-bfd8-9170af3cb8f3 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:16.306819 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bbf22-640e-4598-bfd8-9170af3cb8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:16.326472 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-332bbf22-640e-4598-bfd8-9170af3cb8f3 None None] MFA Rules not processed for user `bbd9bbb083ab4f5e84e135ec5103cdae`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:16.327672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-332bbf22-640e-4598-bfd8-9170af3cb8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:16.382741 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1229/2478] 10.222.0.59 () {58 vars in 967 bytes} [Fri Aug 26 01:00:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 01:00:16.394993 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332bbf22-640e-4598-bfd8-9170af3cb8f3 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:16.396361 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-332bbf22-640e-4598-bfd8-9170af3cb8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:16.413820 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-98eed10b-98af-4fad-9a52-53d5d0127ebf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbdfa897dd3d4070a8cdc22548d76350'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:16.414795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-98eed10b-98af-4fad-9a52-53d5d0127ebf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbdfa897dd3d4070a8cdc22548d76350'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:16.428757 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:16.430550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-98eed10b-98af-4fad-9a52-53d5d0127ebf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbdfa897dd3d4070a8cdc22548d76350'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:16.467938 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332bbf22-640e-4598-bfd8-9170af3cb8f3 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:16.473289 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc52c890-a1f9-40dc-9e71-3bce2e04f426 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f700502671446c797a566f07398a9fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:16.473902 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52c890-a1f9-40dc-9e71-3bce2e04f426 service swift] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:16.474184 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52c890-a1f9-40dc-9e71-3bce2e04f426 service swift] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:16.474496 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52c890-a1f9-40dc-9e71-3bce2e04f426 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:16.476282 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-cc52c890-a1f9-40dc-9e71-3bce2e04f426 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:16.487329 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc52c890-a1f9-40dc-9e71-3bce2e04f426 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:16.489780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc52c890-a1f9-40dc-9e71-3bce2e04f426 service swift] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:16.497807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1230/2479] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Aug 26 01:00:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 01:00:16.511141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-98eed10b-98af-4fad-9a52-53d5d0127ebf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbdfa897dd3d4070a8cdc22548d76350'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:16.511777 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-98eed10b-98af-4fad-9a52-53d5d0127ebf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbdfa897dd3d4070a8cdc22548d76350'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:16.529990 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1250/2480] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:00:16 2022] DELETE /identity/v3/users/dbdfa897dd3d4070a8cdc22548d76350 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:16.992675 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc52c890-a1f9-40dc-9e71-3bce2e04f426 service swift] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:16.994304 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-cc52c890-a1f9-40dc-9e71-3bce2e04f426 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:17.008674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc52c890-a1f9-40dc-9e71-3bce2e04f426 service swift] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:17.068422 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd2951-2605-4beb-afe4-54525002c818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:17.069086 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd2951-2605-4beb-afe4-54525002c818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:17.069370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd2951-2605-4beb-afe4-54525002c818 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:17.069648 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd2951-2605-4beb-afe4-54525002c818 admin admin] PATH_INFO: `/v3/projects/5a61bcb96f00420aa0ec9d8ec3a1fffa` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:17.070797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd2951-2605-4beb-afe4-54525002c818 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a61bcb96f00420aa0ec9d8ec3a1fffa)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:17.072764 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd2951-2605-4beb-afe4-54525002c818 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:17.195781 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-b2bd2951-2605-4beb-afe4-54525002c818 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a61bcb96f00420aa0ec9d8ec3a1fffa', 'request_id': 'req-b2bd2951-2605-4beb-afe4-54525002c818'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:17.305619 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-b2bd2951-2605-4beb-afe4-54525002c818 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a61bcb96f00420aa0ec9d8ec3a1fffa', 'request_id': 'req-b2bd2951-2605-4beb-afe4-54525002c818'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:17.314298 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-b2bd2951-2605-4beb-afe4-54525002c818 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a61bcb96f00420aa0ec9d8ec3a1fffa', 'request_id': 'req-b2bd2951-2605-4beb-afe4-54525002c818'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:17.328436 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-b2bd2951-2605-4beb-afe4-54525002c818 admin admin] The token cache is being invalidate because project 5a61bcb96f00420aa0ec9d8ec3a1fffa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:00:17.328751 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-b2bd2951-2605-4beb-afe4-54525002c818 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:17.329824 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-b2bd2951-2605-4beb-afe4-54525002c818 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:17.331725 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1231/2481] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:00:16 2022] DELETE /identity/v3/projects/5a61bcb96f00420aa0ec9d8ec3a1fffa => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:17.336652 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbfdda-2b2e-439f-a1a0-f3169073ce84 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:17.337154 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbfdda-2b2e-439f-a1a0-f3169073ce84 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:17.337252 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbfdda-2b2e-439f-a1a0-f3169073ce84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:17.357954 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-93fbfdda-2b2e-439f-a1a0-f3169073ce84 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:17.359186 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-93fbfdda-2b2e-439f-a1a0-f3169073ce84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:17.422615 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1251/2482] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 01:00:17.437871 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d28464-1636-46a7-b48e-0a1f039d6dba None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:17.438428 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d28464-1636-46a7-b48e-0a1f039d6dba None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:17.438428 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d28464-1636-46a7-b48e-0a1f039d6dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:17.460299 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-a7d28464-1636-46a7-b48e-0a1f039d6dba None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:17.461469 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-a7d28464-1636-46a7-b48e-0a1f039d6dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:17.469362 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1232/2483] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:00:17.474248 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbfdda-2b2e-439f-a1a0-f3169073ce84 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:17.484037 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbfdda-2b2e-439f-a1a0-f3169073ce84 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:17.489210 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27cf7c-afc7-4658-99eb-c0bb797e2a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:17.489820 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27cf7c-afc7-4658-99eb-c0bb797e2a7c admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:17.490103 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27cf7c-afc7-4658-99eb-c0bb797e2a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:17.490379 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27cf7c-afc7-4658-99eb-c0bb797e2a7c admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:17.491199 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c27cf7c-afc7-4658-99eb-c0bb797e2a7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 01:00:17.491494 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c27cf7c-afc7-4658-99eb-c0bb797e2a7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:17.493153 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c27cf7c-afc7-4658-99eb-c0bb797e2a7c admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:17.505659 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1252/2484] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 01:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 01:00:17.509986 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d28464-1636-46a7-b48e-0a1f039d6dba None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:17.519526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d28464-1636-46a7-b48e-0a1f039d6dba None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:17.524272 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd595e-252b-4b20-bb49-9b4e6e8fd2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:17.524911 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd595e-252b-4b20-bb49-9b4e6e8fd2cf admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:17.525189 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd595e-252b-4b20-bb49-9b4e6e8fd2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:17.525457 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd595e-252b-4b20-bb49-9b4e6e8fd2cf admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:17.526095 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd595e-252b-4b20-bb49-9b4e6e8fd2cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:17.528007 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd595e-252b-4b20-bb49-9b4e6e8fd2cf admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:17.644097 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1233/2485] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:00:17 2022] POST /identity/v3/projects => generated 363 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:00:17.648376 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27cf7c-afc7-4658-99eb-c0bb797e2a7c admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:17.659273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27cf7c-afc7-4658-99eb-c0bb797e2a7c admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:17.702668 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:17.717655 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f8346-5c28-43ca-81ac-827ae902c003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:17.718210 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f8346-5c28-43ca-81ac-827ae902c003 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:17.718493 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f8346-5c28-43ca-81ac-827ae902c003 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:17.718769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f8346-5c28-43ca-81ac-827ae902c003 admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:17.719396 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f8346-5c28-43ca-81ac-827ae902c003 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:17.721901 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f8346-5c28-43ca-81ac-827ae902c003 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:17.808811 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1253/2486] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:00:17 2022] POST /identity/v3/users => generated 282 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:00:17.812870 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd595e-252b-4b20-bb49-9b4e6e8fd2cf admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:17.823179 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd595e-252b-4b20-bb49-9b4e6e8fd2cf admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:17.827834 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8559638-680a-447a-b33c-eac176bd7143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:17.828425 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8559638-680a-447a-b33c-eac176bd7143 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:17.828736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8559638-680a-447a-b33c-eac176bd7143 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:17.829518 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8559638-680a-447a-b33c-eac176bd7143 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:17.830157 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8559638-680a-447a-b33c-eac176bd7143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:17.831740 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8559638-680a-447a-b33c-eac176bd7143 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:17.841442 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1234/2487] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:00:17.849297 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f8346-5c28-43ca-81ac-827ae902c003 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:17.859657 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f8346-5c28-43ca-81ac-827ae902c003 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:17.864949 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa2f340-1041-47be-bb79-58df4eb08de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:17.865728 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa2f340-1041-47be-bb79-58df4eb08de5 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:17.866431 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa2f340-1041-47be-bb79-58df4eb08de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:17.866758 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa2f340-1041-47be-bb79-58df4eb08de5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:17.867452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa2f340-1041-47be-bb79-58df4eb08de5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:17.869060 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa2f340-1041-47be-bb79-58df4eb08de5 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:17.881158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1254/2488] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:00:17.885577 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8559638-680a-447a-b33c-eac176bd7143 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:17.895739 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8559638-680a-447a-b33c-eac176bd7143 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:17.901300 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968645e9-1006-43e8-b45d-45b6a3d53ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:17.901948 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968645e9-1006-43e8-b45d-45b6a3d53ae8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:17.909660 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968645e9-1006-43e8-b45d-45b6a3d53ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:17.909660 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968645e9-1006-43e8-b45d-45b6a3d53ae8 admin admin] PATH_INFO: `/v3/projects/4967d99c6ad44d4fb432cf68ab2976ae/users/1b52fd0bd4b9460486e8ad0fab262f23/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:17.909660 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:17.917670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968645e9-1006-43e8-b45d-45b6a3d53ae8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4967d99c6ad44d4fb432cf68ab2976ae, user_id=1b52fd0bd4b9460486e8ad0fab262f23, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:17.920328 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968645e9-1006-43e8-b45d-45b6a3d53ae8 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:17.980116 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1235/2489] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:00:17 2022] PUT /identity/v3/projects/4967d99c6ad44d4fb432cf68ab2976ae/users/1b52fd0bd4b9460486e8ad0fab262f23/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:17.993510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb154151-c290-4cdc-bf3b-3555b1e72404 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:17.994030 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb154151-c290-4cdc-bf3b-3555b1e72404 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:17.994515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb154151-c290-4cdc-bf3b-3555b1e72404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:18.032171 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-bb154151-c290-4cdc-bf3b-3555b1e72404 None None] MFA Rules not processed for user `1b52fd0bd4b9460486e8ad0fab262f23`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:18.033536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-bb154151-c290-4cdc-bf3b-3555b1e72404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:18.099086 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1255/2490] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:00:18.103549 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968645e9-1006-43e8-b45d-45b6a3d53ae8 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:18.165991 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968645e9-1006-43e8-b45d-45b6a3d53ae8 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:18.170800 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812796c-2698-4a03-9b63-5df833400426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:18.171409 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812796c-2698-4a03-9b63-5df833400426 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:18.171691 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812796c-2698-4a03-9b63-5df833400426 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:18.171969 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812796c-2698-4a03-9b63-5df833400426 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:18.172650 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f812796c-2698-4a03-9b63-5df833400426 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:18.175086 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f812796c-2698-4a03-9b63-5df833400426 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:18.261099 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1236/2491] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:00:18 2022] POST /identity/v3/projects => generated 363 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:00:18.265107 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb154151-c290-4cdc-bf3b-3555b1e72404 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:18.275612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb154151-c290-4cdc-bf3b-3555b1e72404 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:18.339555 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea3ee9-88d1-4a33-8d33-856516cd0f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:18.340247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea3ee9-88d1-4a33-8d33-856516cd0f2e admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:18.340432 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea3ee9-88d1-4a33-8d33-856516cd0f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:18.340745 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea3ee9-88d1-4a33-8d33-856516cd0f2e admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:18.341354 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea3ee9-88d1-4a33-8d33-856516cd0f2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:18.343301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea3ee9-88d1-4a33-8d33-856516cd0f2e admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:18.431920 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1256/2492] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:00:18 2022] POST /identity/v3/users => generated 288 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:00:18.435583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812796c-2698-4a03-9b63-5df833400426 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:18.446176 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812796c-2698-4a03-9b63-5df833400426 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:18.450860 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb7f6c-f376-4e22-a4c4-dd1eac15974b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:18.451462 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb7f6c-f376-4e22-a4c4-dd1eac15974b admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:18.451744 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb7f6c-f376-4e22-a4c4-dd1eac15974b admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:18.452019 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb7f6c-f376-4e22-a4c4-dd1eac15974b admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:18.452713 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb7f6c-f376-4e22-a4c4-dd1eac15974b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:18.454302 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb7f6c-f376-4e22-a4c4-dd1eac15974b admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:18.465389 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1237/2493] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:00:18.470265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea3ee9-88d1-4a33-8d33-856516cd0f2e admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:18.479956 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea3ee9-88d1-4a33-8d33-856516cd0f2e admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:18.484788 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ffded-da68-420d-8881-2b83456f6cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:18.486092 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ffded-da68-420d-8881-2b83456f6cc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:18.486375 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ffded-da68-420d-8881-2b83456f6cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:18.486724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ffded-da68-420d-8881-2b83456f6cc2 admin admin] PATH_INFO: `/v3/projects/d5970495fadc405da65e60bd1597830d/users/6f3e5cf6cbd64a19acbe9f3beded4d4a/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:18.501887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74ffded-da68-420d-8881-2b83456f6cc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5970495fadc405da65e60bd1597830d, user_id=6f3e5cf6cbd64a19acbe9f3beded4d4a, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:18.504669 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74ffded-da68-420d-8881-2b83456f6cc2 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:18.614546 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1257/2494] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:00:18 2022] PUT /identity/v3/projects/d5970495fadc405da65e60bd1597830d/users/6f3e5cf6cbd64a19acbe9f3beded4d4a/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:18.618390 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb7f6c-f376-4e22-a4c4-dd1eac15974b admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:18.628295 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb7f6c-f376-4e22-a4c4-dd1eac15974b admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:18.684266 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ef05f-d730-41bf-97a5-689030bd9a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:18.684919 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ef05f-d730-41bf-97a5-689030bd9a28 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:18.685192 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ef05f-d730-41bf-97a5-689030bd9a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:18.685460 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ef05f-d730-41bf-97a5-689030bd9a28 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:18.686090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ef05f-d730-41bf-97a5-689030bd9a28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:18.687664 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ef05f-d730-41bf-97a5-689030bd9a28 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:18.697019 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1238/2495] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:00:18.701199 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ffded-da68-420d-8881-2b83456f6cc2 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:18.712030 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ffded-da68-420d-8881-2b83456f6cc2 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:18.716849 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa6738-fc42-4077-b943-d92837dfb1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:18.717564 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa6738-fc42-4077-b943-d92837dfb1a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:18.717763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa6738-fc42-4077-b943-d92837dfb1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:18.718031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa6738-fc42-4077-b943-d92837dfb1a7 admin admin] PATH_INFO: `/v3/projects/d5970495fadc405da65e60bd1597830d/users/6f3e5cf6cbd64a19acbe9f3beded4d4a/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:18.720831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aa6738-fc42-4077-b943-d92837dfb1a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5970495fadc405da65e60bd1597830d, user_id=6f3e5cf6cbd64a19acbe9f3beded4d4a, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:18.723623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aa6738-fc42-4077-b943-d92837dfb1a7 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:18.734768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1258/2496] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:00:18 2022] PUT /identity/v3/projects/d5970495fadc405da65e60bd1597830d/users/6f3e5cf6cbd64a19acbe9f3beded4d4a/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 01:00:18.745477 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a627a36-7963-4575-9ebe-74cf0053c397 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:18.745761 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a627a36-7963-4575-9ebe-74cf0053c397 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:18.746039 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a627a36-7963-4575-9ebe-74cf0053c397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:18.777099 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-7a627a36-7963-4575-9ebe-74cf0053c397 None None] MFA Rules not processed for user `6f3e5cf6cbd64a19acbe9f3beded4d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:18.778225 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-7a627a36-7963-4575-9ebe-74cf0053c397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:18.826152 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:18.850871 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1239/2497] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:18 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 01:00:18.862367 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa6738-fc42-4077-b943-d92837dfb1a7 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:18.864051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-01aa6738-fc42-4077-b943-d92837dfb1a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:18.870444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:18.925032 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa6738-fc42-4077-b943-d92837dfb1a7 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:18.929246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c49af4-87f5-4c28-ba77-efc385f56d9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:18.929853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c49af4-87f5-4c28-ba77-efc385f56d9d service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:18.930133 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c49af4-87f5-4c28-ba77-efc385f56d9d service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:18.930406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c49af4-87f5-4c28-ba77-efc385f56d9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:18.940441 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c49af4-87f5-4c28-ba77-efc385f56d9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:18.942005 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c49af4-87f5-4c28-ba77-efc385f56d9d service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:18.951694 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1259/2498] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:00:18 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:00:19.236371 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a627a36-7963-4575-9ebe-74cf0053c397 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:19.245831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a627a36-7963-4575-9ebe-74cf0053c397 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:19.249397 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f272745-9442-4b48-b930-8c77cb7ec4d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:19.251822 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f272745-9442-4b48-b930-8c77cb7ec4d5 service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:19.251822 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f272745-9442-4b48-b930-8c77cb7ec4d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:19.251822 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f272745-9442-4b48-b930-8c77cb7ec4d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:19.306149 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f272745-9442-4b48-b930-8c77cb7ec4d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:19.307547 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f272745-9442-4b48-b930-8c77cb7ec4d5 service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:19.317214 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1240/2499] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:00:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:00:28.963309 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=24 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:29.312730 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=37 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:31.340209 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c49af4-87f5-4c28-ba77-efc385f56d9d service cinder] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:31.402510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c49af4-87f5-4c28-ba77-efc385f56d9d service cinder] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:31.407095 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9b032-8ee5-46b8-90ff-02e9342afed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:31.407765 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9b032-8ee5-46b8-90ff-02e9342afed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:31.408048 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9b032-8ee5-46b8-90ff-02e9342afed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:31.408317 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9b032-8ee5-46b8-90ff-02e9342afed1 admin admin] PATH_INFO: `/v3/users/1b52fd0bd4b9460486e8ad0fab262f23` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:31.410069 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d9b032-8ee5-46b8-90ff-02e9342afed1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b52fd0bd4b9460486e8ad0fab262f23)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:31.412009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d9b032-8ee5-46b8-90ff-02e9342afed1 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:31.627093 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-75d9b032-8ee5-46b8-90ff-02e9342afed1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b52fd0bd4b9460486e8ad0fab262f23'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:31.628264 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-75d9b032-8ee5-46b8-90ff-02e9342afed1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b52fd0bd4b9460486e8ad0fab262f23'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:31.640610 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-75d9b032-8ee5-46b8-90ff-02e9342afed1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b52fd0bd4b9460486e8ad0fab262f23'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:31.710360 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-75d9b032-8ee5-46b8-90ff-02e9342afed1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b52fd0bd4b9460486e8ad0fab262f23'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:31.710921 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-75d9b032-8ee5-46b8-90ff-02e9342afed1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b52fd0bd4b9460486e8ad0fab262f23'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:31.723637 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1260/2500] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:00:31 2022] DELETE /identity/v3/users/1b52fd0bd4b9460486e8ad0fab262f23 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:31.728182 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b5dd7-ef32-45de-8dde-656ed7355ec8 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:31.728684 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b5dd7-ef32-45de-8dde-656ed7355ec8 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:31.728784 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b5dd7-ef32-45de-8dde-656ed7355ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:31.747475 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-545b5dd7-ef32-45de-8dde-656ed7355ec8 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:31.748696 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-545b5dd7-ef32-45de-8dde-656ed7355ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:31.808681 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1241/2501] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:00:31.818987 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9b032-8ee5-46b8-90ff-02e9342afed1 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:31.820421 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-75d9b032-8ee5-46b8-90ff-02e9342afed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:31.878532 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9b032-8ee5-46b8-90ff-02e9342afed1 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:31.882254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7ede2-d149-455d-b357-7182e46a1a85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:31.882988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7ede2-d149-455d-b357-7182e46a1a85 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:31.883272 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7ede2-d149-455d-b357-7182e46a1a85 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:31.883589 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7ede2-d149-455d-b357-7182e46a1a85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:31.895394 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a7ede2-d149-455d-b357-7182e46a1a85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:31.896983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a7ede2-d149-455d-b357-7182e46a1a85 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:31.906827 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1261/2502] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:00:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:00:32.386904 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b5dd7-ef32-45de-8dde-656ed7355ec8 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:32.388365 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-545b5dd7-ef32-45de-8dde-656ed7355ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:32.401267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b5dd7-ef32-45de-8dde-656ed7355ec8 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:32.406005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b484205-1d2f-4d1b-83ea-ba8b93538209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:32.406651 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b484205-1d2f-4d1b-83ea-ba8b93538209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:32.406938 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b484205-1d2f-4d1b-83ea-ba8b93538209 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:32.407221 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b484205-1d2f-4d1b-83ea-ba8b93538209 admin admin] PATH_INFO: `/v3/projects/4967d99c6ad44d4fb432cf68ab2976ae` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:32.408517 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b484205-1d2f-4d1b-83ea-ba8b93538209 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4967d99c6ad44d4fb432cf68ab2976ae)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:32.410328 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b484205-1d2f-4d1b-83ea-ba8b93538209 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:32.535328 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1b484205-1d2f-4d1b-83ea-ba8b93538209 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4967d99c6ad44d4fb432cf68ab2976ae', 'request_id': 'req-1b484205-1d2f-4d1b-83ea-ba8b93538209'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:32.601940 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1b484205-1d2f-4d1b-83ea-ba8b93538209 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4967d99c6ad44d4fb432cf68ab2976ae', 'request_id': 'req-1b484205-1d2f-4d1b-83ea-ba8b93538209'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:32.608440 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1b484205-1d2f-4d1b-83ea-ba8b93538209 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4967d99c6ad44d4fb432cf68ab2976ae', 'request_id': 'req-1b484205-1d2f-4d1b-83ea-ba8b93538209'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:32.618724 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1b484205-1d2f-4d1b-83ea-ba8b93538209 admin admin] The token cache is being invalidate because project 4967d99c6ad44d4fb432cf68ab2976ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:00:32.619008 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1b484205-1d2f-4d1b-83ea-ba8b93538209 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:32.619652 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1b484205-1d2f-4d1b-83ea-ba8b93538209 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:32.621148 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1242/2503] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:00:32 2022] DELETE /identity/v3/projects/4967d99c6ad44d4fb432cf68ab2976ae => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:32.625316 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7ede2-d149-455d-b357-7182e46a1a85 service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:32.635735 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7ede2-d149-455d-b357-7182e46a1a85 service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:32.653812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:32.694313 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadbb4d9-0f42-4e54-99d8-7ae40bdd41f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:32.695394 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbb4d9-0f42-4e54-99d8-7ae40bdd41f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:32.695806 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbb4d9-0f42-4e54-99d8-7ae40bdd41f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:32.696020 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbb4d9-0f42-4e54-99d8-7ae40bdd41f0 admin admin] PATH_INFO: `/v3/users/6f3e5cf6cbd64a19acbe9f3beded4d4a` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:32.697203 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadbb4d9-0f42-4e54-99d8-7ae40bdd41f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f3e5cf6cbd64a19acbe9f3beded4d4a)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:32.699156 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadbb4d9-0f42-4e54-99d8-7ae40bdd41f0 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:32.920171 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-cadbb4d9-0f42-4e54-99d8-7ae40bdd41f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f3e5cf6cbd64a19acbe9f3beded4d4a'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:32.920858 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-cadbb4d9-0f42-4e54-99d8-7ae40bdd41f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f3e5cf6cbd64a19acbe9f3beded4d4a'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:32.932723 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-cadbb4d9-0f42-4e54-99d8-7ae40bdd41f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f3e5cf6cbd64a19acbe9f3beded4d4a'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:33.030905 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-cadbb4d9-0f42-4e54-99d8-7ae40bdd41f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f3e5cf6cbd64a19acbe9f3beded4d4a'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:33.031538 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-cadbb4d9-0f42-4e54-99d8-7ae40bdd41f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f3e5cf6cbd64a19acbe9f3beded4d4a'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:33.045285 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1262/2504] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:00:32 2022] DELETE /identity/v3/users/6f3e5cf6cbd64a19acbe9f3beded4d4a => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:33.449456 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b484205-1d2f-4d1b-83ea-ba8b93538209 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:33.450892 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-1b484205-1d2f-4d1b-83ea-ba8b93538209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:33.465404 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b484205-1d2f-4d1b-83ea-ba8b93538209 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:33.521953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22629284-01c3-4c54-86e4-d7734148277f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:33.522577 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22629284-01c3-4c54-86e4-d7734148277f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:33.522857 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22629284-01c3-4c54-86e4-d7734148277f admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:33.523126 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22629284-01c3-4c54-86e4-d7734148277f admin admin] PATH_INFO: `/v3/projects/d5970495fadc405da65e60bd1597830d` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:33.524220 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22629284-01c3-4c54-86e4-d7734148277f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5970495fadc405da65e60bd1597830d)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:33.526142 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22629284-01c3-4c54-86e4-d7734148277f admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:33.620388 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:33.636540 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-22629284-01c3-4c54-86e4-d7734148277f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5970495fadc405da65e60bd1597830d', 'request_id': 'req-22629284-01c3-4c54-86e4-d7734148277f'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:33.721795 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-22629284-01c3-4c54-86e4-d7734148277f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5970495fadc405da65e60bd1597830d', 'request_id': 'req-22629284-01c3-4c54-86e4-d7734148277f'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:33.728211 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-22629284-01c3-4c54-86e4-d7734148277f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5970495fadc405da65e60bd1597830d', 'request_id': 'req-22629284-01c3-4c54-86e4-d7734148277f'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:33.739089 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-22629284-01c3-4c54-86e4-d7734148277f admin admin] The token cache is being invalidate because project d5970495fadc405da65e60bd1597830d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:00:33.739375 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-22629284-01c3-4c54-86e4-d7734148277f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:33.740003 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-22629284-01c3-4c54-86e4-d7734148277f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:33.741499 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1243/2505] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:00:33 2022] DELETE /identity/v3/projects/d5970495fadc405da65e60bd1597830d => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:33.747473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f9e25-6c47-483b-a230-611758e8d85c None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:33.747971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f9e25-6c47-483b-a230-611758e8d85c None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:33.748073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f9e25-6c47-483b-a230-611758e8d85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:33.768145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-e85f9e25-6c47-483b-a230-611758e8d85c None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:33.769378 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-e85f9e25-6c47-483b-a230-611758e8d85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:33.829082 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1263/2506] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 01:00:33.837042 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e1c79-4afb-48e8-94b5-8d84b13a5cec None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:33.837344 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e1c79-4afb-48e8-94b5-8d84b13a5cec None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:33.837626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e1c79-4afb-48e8-94b5-8d84b13a5cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:33.857303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-d71e1c79-4afb-48e8-94b5-8d84b13a5cec None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:33.858510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-d71e1c79-4afb-48e8-94b5-8d84b13a5cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:33.866926 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1244/2507] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:00:33.871322 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f9e25-6c47-483b-a230-611758e8d85c None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:33.881257 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f9e25-6c47-483b-a230-611758e8d85c None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:33.886214 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1f837-0296-4fd9-934e-a4e650a174b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:33.886824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1f837-0296-4fd9-934e-a4e650a174b5 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:33.887104 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1f837-0296-4fd9-934e-a4e650a174b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:33.887377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1f837-0296-4fd9-934e-a4e650a174b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:33.888048 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1f837-0296-4fd9-934e-a4e650a174b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 01:00:33.888350 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1f837-0296-4fd9-934e-a4e650a174b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:33.889985 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1f837-0296-4fd9-934e-a4e650a174b5 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:33.901898 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1264/2508] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 01:00:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 01:00:33.906176 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71e1c79-4afb-48e8-94b5-8d84b13a5cec None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:33.916226 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71e1c79-4afb-48e8-94b5-8d84b13a5cec None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:33.920968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571bd108-1e92-4114-94db-0302233e8804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:33.921564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bd108-1e92-4114-94db-0302233e8804 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:33.921846 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bd108-1e92-4114-94db-0302233e8804 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:33.922120 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bd108-1e92-4114-94db-0302233e8804 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:33.922758 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571bd108-1e92-4114-94db-0302233e8804 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:33.925213 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571bd108-1e92-4114-94db-0302233e8804 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:34.049324 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1245/2509] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:00:33 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:00:34.054334 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1f837-0296-4fd9-934e-a4e650a174b5 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:34.064686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1f837-0296-4fd9-934e-a4e650a174b5 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:34.091093 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:34.122567 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c11b3-530a-4a28-9299-5637f6e6788c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:34.123177 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c11b3-530a-4a28-9299-5637f6e6788c admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:34.123460 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c11b3-530a-4a28-9299-5637f6e6788c admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:34.123732 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c11b3-530a-4a28-9299-5637f6e6788c admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:34.124369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63c11b3-530a-4a28-9299-5637f6e6788c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:34.126328 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63c11b3-530a-4a28-9299-5637f6e6788c admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:34.236797 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1265/2510] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:00:34 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 01:00:34.242471 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571bd108-1e92-4114-94db-0302233e8804 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:34.252695 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571bd108-1e92-4114-94db-0302233e8804 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:34.257198 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b466c06-f0a0-4c0d-bb48-60d89c8543ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:34.257802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b466c06-f0a0-4c0d-bb48-60d89c8543ab admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:34.258085 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b466c06-f0a0-4c0d-bb48-60d89c8543ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:34.258359 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b466c06-f0a0-4c0d-bb48-60d89c8543ab admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:34.259002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b466c06-f0a0-4c0d-bb48-60d89c8543ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:34.260638 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b466c06-f0a0-4c0d-bb48-60d89c8543ab admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:34.270930 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1246/2511] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:00:34.274999 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c11b3-530a-4a28-9299-5637f6e6788c admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:34.286184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c11b3-530a-4a28-9299-5637f6e6788c admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:34.291354 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04368eb7-126c-4a14-a3db-3282a1859c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:34.291970 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04368eb7-126c-4a14-a3db-3282a1859c89 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:34.292253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04368eb7-126c-4a14-a3db-3282a1859c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:34.292571 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04368eb7-126c-4a14-a3db-3282a1859c89 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:34.293303 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04368eb7-126c-4a14-a3db-3282a1859c89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:34.294871 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04368eb7-126c-4a14-a3db-3282a1859c89 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:34.305804 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1266/2512] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:00:34.310219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b466c06-f0a0-4c0d-bb48-60d89c8543ab admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:34.320403 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b466c06-f0a0-4c0d-bb48-60d89c8543ab admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:34.326098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904efc35-b18a-4270-8cb1-f381dd24478d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:34.326766 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904efc35-b18a-4270-8cb1-f381dd24478d admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:34.327051 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904efc35-b18a-4270-8cb1-f381dd24478d admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:34.327328 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904efc35-b18a-4270-8cb1-f381dd24478d admin admin] PATH_INFO: `/v3/projects/f56cf64983874ef79375ba181072ea33/users/3f654105c3de4db385db68a9687fad8d/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:34.342586 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904efc35-b18a-4270-8cb1-f381dd24478d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f56cf64983874ef79375ba181072ea33, user_id=3f654105c3de4db385db68a9687fad8d, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:34.346249 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904efc35-b18a-4270-8cb1-f381dd24478d admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:34.419558 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1247/2513] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:00:34 2022] PUT /identity/v3/projects/f56cf64983874ef79375ba181072ea33/users/3f654105c3de4db385db68a9687fad8d/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:34.431296 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9cf3c-d6ec-4bdc-847c-3d89919d05d3 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:34.431584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9cf3c-d6ec-4bdc-847c-3d89919d05d3 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:34.431866 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9cf3c-d6ec-4bdc-847c-3d89919d05d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:34.461534 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-66c9cf3c-d6ec-4bdc-847c-3d89919d05d3 None None] MFA Rules not processed for user `3f654105c3de4db385db68a9687fad8d`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:34.463037 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-66c9cf3c-d6ec-4bdc-847c-3d89919d05d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:34.529374 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1267/2514] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:00:34.534475 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904efc35-b18a-4270-8cb1-f381dd24478d admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:34.596915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904efc35-b18a-4270-8cb1-f381dd24478d admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:34.601782 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe58239-d724-4683-bb1f-33c28a9d6aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:34.602422 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe58239-d724-4683-bb1f-33c28a9d6aeb admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:34.602704 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe58239-d724-4683-bb1f-33c28a9d6aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:34.602980 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe58239-d724-4683-bb1f-33c28a9d6aeb admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:34.603612 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe58239-d724-4683-bb1f-33c28a9d6aeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:34.605592 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe58239-d724-4683-bb1f-33c28a9d6aeb admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:34.683366 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1248/2515] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:00:34 2022] POST /identity/v3/projects => generated 387 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:00:34.687655 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9cf3c-d6ec-4bdc-847c-3d89919d05d3 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:34.698317 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9cf3c-d6ec-4bdc-847c-3d89919d05d3 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:34.756281 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9210084-5d70-4fc8-a03c-4374a3d5218e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:34.757036 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9210084-5d70-4fc8-a03c-4374a3d5218e admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:34.757260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9210084-5d70-4fc8-a03c-4374a3d5218e admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:34.757529 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9210084-5d70-4fc8-a03c-4374a3d5218e admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:34.758171 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9210084-5d70-4fc8-a03c-4374a3d5218e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:34.760090 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9210084-5d70-4fc8-a03c-4374a3d5218e admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:34.848337 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1268/2516] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:00:34 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:00:34.852331 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe58239-d724-4683-bb1f-33c28a9d6aeb admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:34.864087 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe58239-d724-4683-bb1f-33c28a9d6aeb admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:34.869920 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de6b9d-d444-4b9b-8056-7ef992e96f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:34.870807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de6b9d-d444-4b9b-8056-7ef992e96f22 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:34.871122 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de6b9d-d444-4b9b-8056-7ef992e96f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:34.871390 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de6b9d-d444-4b9b-8056-7ef992e96f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:34.872071 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de6b9d-d444-4b9b-8056-7ef992e96f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:34.874133 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de6b9d-d444-4b9b-8056-7ef992e96f22 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:34.885171 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1249/2517] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:00:34.894473 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9210084-5d70-4fc8-a03c-4374a3d5218e admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:34.898972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9210084-5d70-4fc8-a03c-4374a3d5218e admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:34.903565 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10748594-090e-45ad-9b32-ecfac0a31d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:34.904159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10748594-090e-45ad-9b32-ecfac0a31d09 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:34.904437 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10748594-090e-45ad-9b32-ecfac0a31d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:34.904738 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10748594-090e-45ad-9b32-ecfac0a31d09 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:34.905377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10748594-090e-45ad-9b32-ecfac0a31d09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:34.906954 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10748594-090e-45ad-9b32-ecfac0a31d09 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:34.917563 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1269/2518] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:00:34.921625 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de6b9d-d444-4b9b-8056-7ef992e96f22 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:34.932053 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de6b9d-d444-4b9b-8056-7ef992e96f22 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:34.936769 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0432a9ac-8071-4400-bdba-5c6efa86fc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:34.940159 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0432a9ac-8071-4400-bdba-5c6efa86fc10 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:34.940159 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0432a9ac-8071-4400-bdba-5c6efa86fc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:34.940159 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0432a9ac-8071-4400-bdba-5c6efa86fc10 admin admin] PATH_INFO: `/v3/projects/9e329fc725894b0984dc24894a867988/users/2939e228ac7b46a89b5c6b9fb13f8254/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:34.952114 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0432a9ac-8071-4400-bdba-5c6efa86fc10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e329fc725894b0984dc24894a867988, user_id=2939e228ac7b46a89b5c6b9fb13f8254, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:34.954868 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0432a9ac-8071-4400-bdba-5c6efa86fc10 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:35.060733 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:35.066024 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1250/2519] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:00:34 2022] PUT /identity/v3/projects/9e329fc725894b0984dc24894a867988/users/2939e228ac7b46a89b5c6b9fb13f8254/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 01:00:35.076381 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325ea64-d840-444b-b5a1-24cf5804f7f5 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:35.076690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325ea64-d840-444b-b5a1-24cf5804f7f5 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:35.076969 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325ea64-d840-444b-b5a1-24cf5804f7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:35.104101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-5325ea64-d840-444b-b5a1-24cf5804f7f5 None None] MFA Rules not processed for user `2939e228ac7b46a89b5c6b9fb13f8254`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:35.105287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-5325ea64-d840-444b-b5a1-24cf5804f7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:35.109320 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:35.171718 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1270/2520] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 01:00:35.176815 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0432a9ac-8071-4400-bdba-5c6efa86fc10 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:35.237779 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0432a9ac-8071-4400-bdba-5c6efa86fc10 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:35.242553 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2740cf7b-3329-46ac-887f-58ff5780b700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:35.243185 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740cf7b-3329-46ac-887f-58ff5780b700 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:35.243468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740cf7b-3329-46ac-887f-58ff5780b700 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:35.243742 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740cf7b-3329-46ac-887f-58ff5780b700 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:35.244369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2740cf7b-3329-46ac-887f-58ff5780b700 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:35.246385 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2740cf7b-3329-46ac-887f-58ff5780b700 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:35.338376 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1251/2521] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:00:35 2022] POST /identity/v3/projects => generated 387 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:00:35.342476 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325ea64-d840-444b-b5a1-24cf5804f7f5 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:35.354359 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325ea64-d840-444b-b5a1-24cf5804f7f5 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:35.417834 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7abc01-ae9c-47c1-90f7-5a5dd9ec4ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:35.418455 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7abc01-ae9c-47c1-90f7-5a5dd9ec4ec1 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:35.418738 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7abc01-ae9c-47c1-90f7-5a5dd9ec4ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:35.419014 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7abc01-ae9c-47c1-90f7-5a5dd9ec4ec1 admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:35.419799 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7abc01-ae9c-47c1-90f7-5a5dd9ec4ec1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:35.421767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7abc01-ae9c-47c1-90f7-5a5dd9ec4ec1 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:35.528073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1271/2522] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:00:35 2022] POST /identity/v3/users => generated 300 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 01:00:35.532175 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2740cf7b-3329-46ac-887f-58ff5780b700 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:35.543463 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2740cf7b-3329-46ac-887f-58ff5780b700 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:35.548066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87863e21-9d35-4660-a122-ccf8a7665307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:35.548702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87863e21-9d35-4660-a122-ccf8a7665307 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:35.549150 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87863e21-9d35-4660-a122-ccf8a7665307 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:35.550077 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87863e21-9d35-4660-a122-ccf8a7665307 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:35.550670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87863e21-9d35-4660-a122-ccf8a7665307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:35.552263 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87863e21-9d35-4660-a122-ccf8a7665307 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:35.563464 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1252/2523] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:00:35.567522 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7abc01-ae9c-47c1-90f7-5a5dd9ec4ec1 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:35.578411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7abc01-ae9c-47c1-90f7-5a5dd9ec4ec1 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:35.583143 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf5647-b27c-453c-8cbf-f90ac492e52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:35.583800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf5647-b27c-453c-8cbf-f90ac492e52e admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:35.584083 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf5647-b27c-453c-8cbf-f90ac492e52e admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:35.584358 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf5647-b27c-453c-8cbf-f90ac492e52e admin admin] PATH_INFO: `/v3/projects/90c29a49bd5e4d48a0c5ba8b88b26734/users/af382b714d29494c972b7c555c2f7a55/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:35.599894 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf5647-b27c-453c-8cbf-f90ac492e52e admin admin] RBAC: Authorizing `identity:create_grant(project_id=90c29a49bd5e4d48a0c5ba8b88b26734, user_id=af382b714d29494c972b7c555c2f7a55, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:35.602588 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf5647-b27c-453c-8cbf-f90ac492e52e admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:35.672427 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1272/2524] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:00:35 2022] PUT /identity/v3/projects/90c29a49bd5e4d48a0c5ba8b88b26734/users/af382b714d29494c972b7c555c2f7a55/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 01:00:35.676964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87863e21-9d35-4660-a122-ccf8a7665307 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:35.687836 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87863e21-9d35-4660-a122-ccf8a7665307 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:35.743593 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78068dfd-e7a5-4950-acad-605fe31f4cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:35.744203 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78068dfd-e7a5-4950-acad-605fe31f4cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:35.744516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78068dfd-e7a5-4950-acad-605fe31f4cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:35.744809 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78068dfd-e7a5-4950-acad-605fe31f4cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:35.745440 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78068dfd-e7a5-4950-acad-605fe31f4cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:35.747027 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78068dfd-e7a5-4950-acad-605fe31f4cc9 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:35.756735 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1253/2525] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:00:35.760963 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf5647-b27c-453c-8cbf-f90ac492e52e admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:35.772221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf5647-b27c-453c-8cbf-f90ac492e52e admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:35.777498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57680ac9-aa49-4a92-bb07-a0c6f116220f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:35.778169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57680ac9-aa49-4a92-bb07-a0c6f116220f admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:35.778450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57680ac9-aa49-4a92-bb07-a0c6f116220f admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:35.778723 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57680ac9-aa49-4a92-bb07-a0c6f116220f admin admin] PATH_INFO: `/v3/projects/90c29a49bd5e4d48a0c5ba8b88b26734/users/af382b714d29494c972b7c555c2f7a55/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:35.780710 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57680ac9-aa49-4a92-bb07-a0c6f116220f admin admin] RBAC: Authorizing `identity:create_grant(project_id=90c29a49bd5e4d48a0c5ba8b88b26734, user_id=af382b714d29494c972b7c555c2f7a55, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:35.783519 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57680ac9-aa49-4a92-bb07-a0c6f116220f admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:35.794899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1273/2526] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:00:35 2022] PUT /identity/v3/projects/90c29a49bd5e4d48a0c5ba8b88b26734/users/af382b714d29494c972b7c555c2f7a55/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:35.805231 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf44afb-1838-4556-be00-f488f042dcfb None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:35.805721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf44afb-1838-4556-be00-f488f042dcfb None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:35.806472 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf44afb-1838-4556-be00-f488f042dcfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:35.834598 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-faf44afb-1838-4556-be00-f488f042dcfb None None] MFA Rules not processed for user `af382b714d29494c972b7c555c2f7a55`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:35.835889 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-faf44afb-1838-4556-be00-f488f042dcfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:35.897018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:35.908465 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1254/2527] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:35 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:00:35.920017 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57680ac9-aa49-4a92-bb07-a0c6f116220f admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:35.921783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-57680ac9-aa49-4a92-bb07-a0c6f116220f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:35.983700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57680ac9-aa49-4a92-bb07-a0c6f116220f admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:35.987387 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957e44e8-ee0e-4fbf-b347-de65084b4ef3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:35.988002 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e44e8-ee0e-4fbf-b347-de65084b4ef3 service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:35.988274 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e44e8-ee0e-4fbf-b347-de65084b4ef3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:35.989111 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e44e8-ee0e-4fbf-b347-de65084b4ef3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:35.998002 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957e44e8-ee0e-4fbf-b347-de65084b4ef3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:35.999445 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957e44e8-ee0e-4fbf-b347-de65084b4ef3 service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:36.005093 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:36.010017 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1274/2528] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:00:35 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:00:37.067601 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf44afb-1838-4556-be00-f488f042dcfb None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:37.069060 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-faf44afb-1838-4556-be00-f488f042dcfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:37.080683 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf44afb-1838-4556-be00-f488f042dcfb None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:37.084178 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd092bf6-f108-44f5-ac6b-a83cad1e508f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:37.084802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd092bf6-f108-44f5-ac6b-a83cad1e508f service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:37.085088 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd092bf6-f108-44f5-ac6b-a83cad1e508f service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:37.085361 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd092bf6-f108-44f5-ac6b-a83cad1e508f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:37.142934 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd092bf6-f108-44f5-ac6b-a83cad1e508f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:37.144409 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd092bf6-f108-44f5-ac6b-a83cad1e508f service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:37.153829 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1255/2529] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:00:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:00:42.223549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957e44e8-ee0e-4fbf-b347-de65084b4ef3 service cinder] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:42.232141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957e44e8-ee0e-4fbf-b347-de65084b4ef3 service cinder] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:42.235814 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa7198-1755-4f9d-ae10-8d265ecfb9aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:42.236410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa7198-1755-4f9d-ae10-8d265ecfb9aa service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:42.236751 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa7198-1755-4f9d-ae10-8d265ecfb9aa service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:42.237021 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa7198-1755-4f9d-ae10-8d265ecfb9aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:42.293882 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befa7198-1755-4f9d-ae10-8d265ecfb9aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:42.295320 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befa7198-1755-4f9d-ae10-8d265ecfb9aa service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:42.304152 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1275/2530] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:00:42 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:00:45.558627 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd092bf6-f108-44f5-ac6b-a83cad1e508f service cinder] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:45.560027 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-bd092bf6-f108-44f5-ac6b-a83cad1e508f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:45.616527 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd092bf6-f108-44f5-ac6b-a83cad1e508f service cinder] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:45.620270 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c9768-814d-45f8-b5a5-feba32eccff5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:45.620901 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c9768-814d-45f8-b5a5-feba32eccff5 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:45.621185 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c9768-814d-45f8-b5a5-feba32eccff5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:45.621460 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c9768-814d-45f8-b5a5-feba32eccff5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:45.623167 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-2a7c9768-814d-45f8-b5a5-feba32eccff5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:45.685516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c9768-814d-45f8-b5a5-feba32eccff5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:45.686969 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c9768-814d-45f8-b5a5-feba32eccff5 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:45.696578 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1256/2531] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:00:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:00:46.395733 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa7198-1755-4f9d-ae10-8d265ecfb9aa service cinder] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:46.407546 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa7198-1755-4f9d-ae10-8d265ecfb9aa service cinder] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:46.413417 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81463378-6706-4d5c-a41f-c99febf2a75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:46.414058 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81463378-6706-4d5c-a41f-c99febf2a75a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:46.414344 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81463378-6706-4d5c-a41f-c99febf2a75a admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:46.414617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81463378-6706-4d5c-a41f-c99febf2a75a admin admin] PATH_INFO: `/v3/users/3f654105c3de4db385db68a9687fad8d` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:46.416377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81463378-6706-4d5c-a41f-c99febf2a75a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f654105c3de4db385db68a9687fad8d)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:46.418372 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81463378-6706-4d5c-a41f-c99febf2a75a admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:46.622657 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-81463378-6706-4d5c-a41f-c99febf2a75a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f654105c3de4db385db68a9687fad8d'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:46.623344 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-81463378-6706-4d5c-a41f-c99febf2a75a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f654105c3de4db385db68a9687fad8d'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:46.636164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-81463378-6706-4d5c-a41f-c99febf2a75a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f654105c3de4db385db68a9687fad8d'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:46.699104 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-81463378-6706-4d5c-a41f-c99febf2a75a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f654105c3de4db385db68a9687fad8d'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:46.699872 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-81463378-6706-4d5c-a41f-c99febf2a75a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f654105c3de4db385db68a9687fad8d'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:46.714078 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1276/2532] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:00:46 2022] DELETE /identity/v3/users/3f654105c3de4db385db68a9687fad8d => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:46.718603 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6949c-f4d4-4bd1-af8f-77b634a38d7b None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:46.719107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6949c-f4d4-4bd1-af8f-77b634a38d7b None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:46.719208 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6949c-f4d4-4bd1-af8f-77b634a38d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:46.738808 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-b9f6949c-f4d4-4bd1-af8f-77b634a38d7b None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:46.740031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-b9f6949c-f4d4-4bd1-af8f-77b634a38d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:46.780856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:46.799942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1257/2533] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:00:46.810925 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81463378-6706-4d5c-a41f-c99febf2a75a admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:46.813958 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-81463378-6706-4d5c-a41f-c99febf2a75a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:46.872590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81463378-6706-4d5c-a41f-c99febf2a75a admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:46.876520 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d981e-8dde-46e1-adfb-80e8c4979546 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:46.877789 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d981e-8dde-46e1-adfb-80e8c4979546 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:46.878159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d981e-8dde-46e1-adfb-80e8c4979546 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:46.878490 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d981e-8dde-46e1-adfb-80e8c4979546 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:46.889759 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6d981e-8dde-46e1-adfb-80e8c4979546 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:46.891299 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6d981e-8dde-46e1-adfb-80e8c4979546 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:46.901925 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1277/2534] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:00:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 01:00:47.387104 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6949c-f4d4-4bd1-af8f-77b634a38d7b None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:47.388647 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-b9f6949c-f4d4-4bd1-af8f-77b634a38d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:47.401997 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6949c-f4d4-4bd1-af8f-77b634a38d7b None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:47.407000 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64888538-e143-484e-8323-f852fd0de417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:47.407915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64888538-e143-484e-8323-f852fd0de417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:47.407915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64888538-e143-484e-8323-f852fd0de417 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:47.408200 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64888538-e143-484e-8323-f852fd0de417 admin admin] PATH_INFO: `/v3/projects/f56cf64983874ef79375ba181072ea33` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:47.409899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64888538-e143-484e-8323-f852fd0de417 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f56cf64983874ef79375ba181072ea33)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:47.411930 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64888538-e143-484e-8323-f852fd0de417 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:47.521542 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-64888538-e143-484e-8323-f852fd0de417 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f56cf64983874ef79375ba181072ea33', 'request_id': 'req-64888538-e143-484e-8323-f852fd0de417'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:47.590209 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-64888538-e143-484e-8323-f852fd0de417 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f56cf64983874ef79375ba181072ea33', 'request_id': 'req-64888538-e143-484e-8323-f852fd0de417'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:47.598011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-64888538-e143-484e-8323-f852fd0de417 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f56cf64983874ef79375ba181072ea33', 'request_id': 'req-64888538-e143-484e-8323-f852fd0de417'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:47.610626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-64888538-e143-484e-8323-f852fd0de417 admin admin] The token cache is being invalidate because project f56cf64983874ef79375ba181072ea33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:00:47.613270 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-64888538-e143-484e-8323-f852fd0de417 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:47.613270 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-64888538-e143-484e-8323-f852fd0de417 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:47.614411 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1258/2535] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:00:47 2022] DELETE /identity/v3/projects/f56cf64983874ef79375ba181072ea33 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:47.618238 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d981e-8dde-46e1-adfb-80e8c4979546 service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:47.627938 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d981e-8dde-46e1-adfb-80e8c4979546 service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:47.637954 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:47.690725 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ab5f4-600b-48b8-b2cb-f573d439db6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:47.691455 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ab5f4-600b-48b8-b2cb-f573d439db6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:47.691897 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ab5f4-600b-48b8-b2cb-f573d439db6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:47.692602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ab5f4-600b-48b8-b2cb-f573d439db6d admin admin] PATH_INFO: `/v3/users/2939e228ac7b46a89b5c6b9fb13f8254` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:47.694504 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ab5f4-600b-48b8-b2cb-f573d439db6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2939e228ac7b46a89b5c6b9fb13f8254)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:47.696938 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ab5f4-600b-48b8-b2cb-f573d439db6d admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:47.895835 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-681ab5f4-600b-48b8-b2cb-f573d439db6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2939e228ac7b46a89b5c6b9fb13f8254'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:47.896318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-681ab5f4-600b-48b8-b2cb-f573d439db6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2939e228ac7b46a89b5c6b9fb13f8254'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:47.911025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-681ab5f4-600b-48b8-b2cb-f573d439db6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2939e228ac7b46a89b5c6b9fb13f8254'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:47.996199 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-681ab5f4-600b-48b8-b2cb-f573d439db6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2939e228ac7b46a89b5c6b9fb13f8254'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:47.996874 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-681ab5f4-600b-48b8-b2cb-f573d439db6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2939e228ac7b46a89b5c6b9fb13f8254'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:48.015919 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1278/2536] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:00:47 2022] DELETE /identity/v3/users/2939e228ac7b46a89b5c6b9fb13f8254 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:48.455894 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64888538-e143-484e-8323-f852fd0de417 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:48.457526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-64888538-e143-484e-8323-f852fd0de417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:48.473532 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64888538-e143-484e-8323-f852fd0de417 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:48.534028 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902a2a7-8f54-4540-8034-ac7ce2458737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:48.534668 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902a2a7-8f54-4540-8034-ac7ce2458737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:48.534949 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902a2a7-8f54-4540-8034-ac7ce2458737 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:48.535264 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902a2a7-8f54-4540-8034-ac7ce2458737 admin admin] PATH_INFO: `/v3/projects/9e329fc725894b0984dc24894a867988` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:48.536444 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902a2a7-8f54-4540-8034-ac7ce2458737 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e329fc725894b0984dc24894a867988)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:48.540896 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902a2a7-8f54-4540-8034-ac7ce2458737 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:48.677908 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c902a2a7-8f54-4540-8034-ac7ce2458737 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e329fc725894b0984dc24894a867988', 'request_id': 'req-c902a2a7-8f54-4540-8034-ac7ce2458737'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:48.750462 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c902a2a7-8f54-4540-8034-ac7ce2458737 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e329fc725894b0984dc24894a867988', 'request_id': 'req-c902a2a7-8f54-4540-8034-ac7ce2458737'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:48.758252 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c902a2a7-8f54-4540-8034-ac7ce2458737 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e329fc725894b0984dc24894a867988', 'request_id': 'req-c902a2a7-8f54-4540-8034-ac7ce2458737'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:48.764798 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:48.772347 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c902a2a7-8f54-4540-8034-ac7ce2458737 admin admin] The token cache is being invalidate because project 9e329fc725894b0984dc24894a867988 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:00:48.772785 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c902a2a7-8f54-4540-8034-ac7ce2458737 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:48.773533 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-c902a2a7-8f54-4540-8034-ac7ce2458737 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:48.775036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1259/2537] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:00:48 2022] DELETE /identity/v3/projects/9e329fc725894b0984dc24894a867988 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:48.779450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ab5f4-600b-48b8-b2cb-f573d439db6d admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:48.781010 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-681ab5f4-600b-48b8-b2cb-f573d439db6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:48.796095 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ab5f4-600b-48b8-b2cb-f573d439db6d admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:48.866330 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369760af-7da7-4c1c-a5a6-16ebebbe2d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:48.867450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369760af-7da7-4c1c-a5a6-16ebebbe2d92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:48.868118 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369760af-7da7-4c1c-a5a6-16ebebbe2d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:48.868621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369760af-7da7-4c1c-a5a6-16ebebbe2d92 admin admin] PATH_INFO: `/v3/users/af382b714d29494c972b7c555c2f7a55` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:48.870027 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369760af-7da7-4c1c-a5a6-16ebebbe2d92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af382b714d29494c972b7c555c2f7a55)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:48.872312 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369760af-7da7-4c1c-a5a6-16ebebbe2d92 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:49.087368 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:49.112159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-369760af-7da7-4c1c-a5a6-16ebebbe2d92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af382b714d29494c972b7c555c2f7a55'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:49.113251 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-369760af-7da7-4c1c-a5a6-16ebebbe2d92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af382b714d29494c972b7c555c2f7a55'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:49.131527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-369760af-7da7-4c1c-a5a6-16ebebbe2d92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af382b714d29494c972b7c555c2f7a55'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:49.253539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-369760af-7da7-4c1c-a5a6-16ebebbe2d92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af382b714d29494c972b7c555c2f7a55'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:49.254343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-369760af-7da7-4c1c-a5a6-16ebebbe2d92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af382b714d29494c972b7c555c2f7a55'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:49.270111 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1279/2538] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:00:48 2022] DELETE /identity/v3/users/af382b714d29494c972b7c555c2f7a55 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:49.583195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902a2a7-8f54-4540-8034-ac7ce2458737 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:49.584794 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-c902a2a7-8f54-4540-8034-ac7ce2458737 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:49.657798 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902a2a7-8f54-4540-8034-ac7ce2458737 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:49.686931 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369760af-7da7-4c1c-a5a6-16ebebbe2d92 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:49.688542 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-369760af-7da7-4c1c-a5a6-16ebebbe2d92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:49.730554 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5e058-afe3-472b-9ec1-c0936eadd467 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2876f4182f0949039b862f053533fe85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:49.733056 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5e058-afe3-472b-9ec1-c0936eadd467 service placement] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:49.733476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5e058-afe3-472b-9ec1-c0936eadd467 service placement] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:49.733940 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5e058-afe3-472b-9ec1-c0936eadd467 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:49.736044 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-27e5e058-afe3-472b-9ec1-c0936eadd467 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:49.749494 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5e058-afe3-472b-9ec1-c0936eadd467 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:49.751119 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5e058-afe3-472b-9ec1-c0936eadd467 service placement] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:49.762079 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1260/2539] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:00:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:00:49.764441 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5e058-afe3-472b-9ec1-c0936eadd467 service placement] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:49.765746 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-27e5e058-afe3-472b-9ec1-c0936eadd467 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:49.766585 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369760af-7da7-4c1c-a5a6-16ebebbe2d92 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:49.773340 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575fdc4-40a3-4657-b377-f0327501e1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:49.773794 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575fdc4-40a3-4657-b377-f0327501e1e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:49.774131 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575fdc4-40a3-4657-b377-f0327501e1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:49.784616 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575fdc4-40a3-4657-b377-f0327501e1e7 admin admin] PATH_INFO: `/v3/users/0175a2b440704d51aa14607a2e20e36a` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:49.784616 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575fdc4-40a3-4657-b377-f0327501e1e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0175a2b440704d51aa14607a2e20e36a)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:49.784616 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575fdc4-40a3-4657-b377-f0327501e1e7 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:49.786057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5e058-afe3-472b-9ec1-c0936eadd467 service placement] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:49.790226 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89fa73-beb7-4d6b-ad42-1ea4e285ccdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:49.790807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89fa73-beb7-4d6b-ad42-1ea4e285ccdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:49.791088 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89fa73-beb7-4d6b-ad42-1ea4e285ccdd admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:49.791391 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89fa73-beb7-4d6b-ad42-1ea4e285ccdd admin admin] PATH_INFO: `/v3/projects/90c29a49bd5e4d48a0c5ba8b88b26734` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:49.793168 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d89fa73-beb7-4d6b-ad42-1ea4e285ccdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=90c29a49bd5e4d48a0c5ba8b88b26734)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:49.795389 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d89fa73-beb7-4d6b-ad42-1ea4e285ccdd admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:49.971560 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-5575fdc4-40a3-4657-b377-f0327501e1e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0175a2b440704d51aa14607a2e20e36a'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:49.972564 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-5575fdc4-40a3-4657-b377-f0327501e1e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0175a2b440704d51aa14607a2e20e36a'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:49.973096 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-9d89fa73-beb7-4d6b-ad42-1ea4e285ccdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90c29a49bd5e4d48a0c5ba8b88b26734', 'request_id': 'req-9d89fa73-beb7-4d6b-ad42-1ea4e285ccdd'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:49.985118 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-5575fdc4-40a3-4657-b377-f0327501e1e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0175a2b440704d51aa14607a2e20e36a'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:50.041367 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-9d89fa73-beb7-4d6b-ad42-1ea4e285ccdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90c29a49bd5e4d48a0c5ba8b88b26734', 'request_id': 'req-9d89fa73-beb7-4d6b-ad42-1ea4e285ccdd'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:50.047859 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-9d89fa73-beb7-4d6b-ad42-1ea4e285ccdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90c29a49bd5e4d48a0c5ba8b88b26734', 'request_id': 'req-9d89fa73-beb7-4d6b-ad42-1ea4e285ccdd'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:50.058183 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-9d89fa73-beb7-4d6b-ad42-1ea4e285ccdd admin admin] The token cache is being invalidate because project 90c29a49bd5e4d48a0c5ba8b88b26734 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:00:50.058470 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-9d89fa73-beb7-4d6b-ad42-1ea4e285ccdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:50.059112 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-9d89fa73-beb7-4d6b-ad42-1ea4e285ccdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:50.060626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1261/2540] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:00:49 2022] DELETE /identity/v3/projects/90c29a49bd5e4d48a0c5ba8b88b26734 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:50.066161 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8cafb6-04f6-43b4-b3bd-1b9cc0718aa9 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:50.066442 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8cafb6-04f6-43b4-b3bd-1b9cc0718aa9 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:50.066716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8cafb6-04f6-43b4-b3bd-1b9cc0718aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:50.085651 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-9f8cafb6-04f6-43b4-b3bd-1b9cc0718aa9 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:50.086887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-9f8cafb6-04f6-43b4-b3bd-1b9cc0718aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:50.095080 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:50.098381 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-5575fdc4-40a3-4657-b377-f0327501e1e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0175a2b440704d51aa14607a2e20e36a'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:50.099095 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-5575fdc4-40a3-4657-b377-f0327501e1e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0175a2b440704d51aa14607a2e20e36a'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:50.112233 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1280/2541] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:00:49 2022] DELETE /identity/v3/users/0175a2b440704d51aa14607a2e20e36a => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:50.145923 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1262/2542] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 01:00:50.154399 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924af90-827d-4d37-b13c-95ea0a8344aa None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:50.154677 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924af90-827d-4d37-b13c-95ea0a8344aa None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:50.154956 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924af90-827d-4d37-b13c-95ea0a8344aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:50.175569 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-a924af90-827d-4d37-b13c-95ea0a8344aa None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:50.176817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a924af90-827d-4d37-b13c-95ea0a8344aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:50.203034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:50.239986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1281/2543] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:00:50.241256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8cafb6-04f6-43b4-b3bd-1b9cc0718aa9 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:50.252548 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8cafb6-04f6-43b4-b3bd-1b9cc0718aa9 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:50.258267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2e4c8-5e95-4b4a-8622-3d1b69e931ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:50.258907 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2e4c8-5e95-4b4a-8622-3d1b69e931ad admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:50.259189 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2e4c8-5e95-4b4a-8622-3d1b69e931ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:50.259460 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2e4c8-5e95-4b4a-8622-3d1b69e931ad admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:50.260117 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2e4c8-5e95-4b4a-8622-3d1b69e931ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 01:00:50.260420 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2e4c8-5e95-4b4a-8622-3d1b69e931ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:50.262103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2e4c8-5e95-4b4a-8622-3d1b69e931ad admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:50.274765 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1263/2544] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 01:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 01:00:50.279239 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924af90-827d-4d37-b13c-95ea0a8344aa None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:50.289226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924af90-827d-4d37-b13c-95ea0a8344aa None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:50.296850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f9152-5d4f-4eb9-95bc-e9176f59f1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:50.296850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f9152-5d4f-4eb9-95bc-e9176f59f1a7 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:50.296850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f9152-5d4f-4eb9-95bc-e9176f59f1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:50.296850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f9152-5d4f-4eb9-95bc-e9176f59f1a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:50.297266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f9152-5d4f-4eb9-95bc-e9176f59f1a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:50.298137 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f9152-5d4f-4eb9-95bc-e9176f59f1a7 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:50.402781 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2e4c8-5e95-4b4a-8622-3d1b69e931ad admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:50.404575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-9fb2e4c8-5e95-4b4a-8622-3d1b69e931ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:50.420797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2e4c8-5e95-4b4a-8622-3d1b69e931ad admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:50.426276 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad044577-26e7-4cb2-862e-c3f024bba6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:50.427718 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad044577-26e7-4cb2-862e-c3f024bba6ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:50.427718 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad044577-26e7-4cb2-862e-c3f024bba6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:50.427718 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad044577-26e7-4cb2-862e-c3f024bba6ec admin admin] PATH_INFO: `/v3/projects/5219060f4c184521919b017e8435eca5` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:50.427989 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1282/2545] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:00:50 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:00:50.431989 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad044577-26e7-4cb2-862e-c3f024bba6ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=5219060f4c184521919b017e8435eca5)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:50.433227 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f9152-5d4f-4eb9-95bc-e9176f59f1a7 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:50.433648 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad044577-26e7-4cb2-862e-c3f024bba6ec admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:50.500249 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f9152-5d4f-4eb9-95bc-e9176f59f1a7 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:50.505686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccdeb84-9b5e-491a-9155-1e77f59044ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:50.506341 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdeb84-9b5e-491a-9155-1e77f59044ba admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:50.506624 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdeb84-9b5e-491a-9155-1e77f59044ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:50.506897 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdeb84-9b5e-491a-9155-1e77f59044ba admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:50.507551 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccdeb84-9b5e-491a-9155-1e77f59044ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:50.509583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccdeb84-9b5e-491a-9155-1e77f59044ba admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:50.538603 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-ad044577-26e7-4cb2-862e-c3f024bba6ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5219060f4c184521919b017e8435eca5', 'request_id': 'req-ad044577-26e7-4cb2-862e-c3f024bba6ec'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:50.607814 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1283/2546] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:00:50 2022] POST /identity/v3/users => generated 288 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 01:00:50.611797 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccdeb84-9b5e-491a-9155-1e77f59044ba admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:50.614471 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-ad044577-26e7-4cb2-862e-c3f024bba6ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5219060f4c184521919b017e8435eca5', 'request_id': 'req-ad044577-26e7-4cb2-862e-c3f024bba6ec'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:50.620676 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-ad044577-26e7-4cb2-862e-c3f024bba6ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5219060f4c184521919b017e8435eca5', 'request_id': 'req-ad044577-26e7-4cb2-862e-c3f024bba6ec'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:50.622555 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccdeb84-9b5e-491a-9155-1e77f59044ba admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:50.631373 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-ad044577-26e7-4cb2-862e-c3f024bba6ec admin admin] The token cache is being invalidate because project 5219060f4c184521919b017e8435eca5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:00:50.631657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-ad044577-26e7-4cb2-862e-c3f024bba6ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:50.632300 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-ad044577-26e7-4cb2-862e-c3f024bba6ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:50.633795 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1264/2547] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:00:50 2022] DELETE /identity/v3/projects/5219060f4c184521919b017e8435eca5 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 01:00:50.638846 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce09f5-87e0-41c0-90d6-1ead74ac6565 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:50.639178 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce09f5-87e0-41c0-90d6-1ead74ac6565 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:50.639453 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce09f5-87e0-41c0-90d6-1ead74ac6565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:50.661565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-bcce09f5-87e0-41c0-90d6-1ead74ac6565 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:50.663108 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-bcce09f5-87e0-41c0-90d6-1ead74ac6565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:50.683086 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c71dea-6e2e-4cc4-9f17-771861994569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:50.683997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c71dea-6e2e-4cc4-9f17-771861994569 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:50.684334 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c71dea-6e2e-4cc4-9f17-771861994569 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:50.684678 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c71dea-6e2e-4cc4-9f17-771861994569 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:50.685345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c71dea-6e2e-4cc4-9f17-771861994569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:50.687589 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c71dea-6e2e-4cc4-9f17-771861994569 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:50.700193 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1284/2548] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 01:00:50.704087 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c71dea-6e2e-4cc4-9f17-771861994569 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:50.707841 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-11c71dea-6e2e-4cc4-9f17-771861994569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:50.721061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c71dea-6e2e-4cc4-9f17-771861994569 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:50.725429 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1265/2549] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 01:00:50.725766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a4110-19a6-4371-98fe-04127236244f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:50.726518 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a4110-19a6-4371-98fe-04127236244f admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:50.726866 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a4110-19a6-4371-98fe-04127236244f admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:50.727206 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a4110-19a6-4371-98fe-04127236244f admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:50.727909 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a4110-19a6-4371-98fe-04127236244f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:50.729553 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a4110-19a6-4371-98fe-04127236244f admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:50.741442 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c9334-8c72-4b8b-993e-661632e7c056 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:50.741728 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c9334-8c72-4b8b-993e-661632e7c056 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:50.742006 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c9334-8c72-4b8b-993e-661632e7c056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:50.745367 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1285/2550] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 01:00:50.746423 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a4110-19a6-4371-98fe-04127236244f admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:50.756572 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a4110-19a6-4371-98fe-04127236244f admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:50.761147 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a0ffb-cfee-405f-8acc-34b54f9c6f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:50.761720 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-477c9334-8c72-4b8b-993e-661632e7c056 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:50.761836 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a0ffb-cfee-405f-8acc-34b54f9c6f9d admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:50.762046 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a0ffb-cfee-405f-8acc-34b54f9c6f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:50.762315 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a0ffb-cfee-405f-8acc-34b54f9c6f9d admin admin] PATH_INFO: `/v3/projects/f1d6b36254474af1ac33f6e07d8b3942/users/fc55962b4ca64a6e9c8f52407da4d763/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:50.762920 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-477c9334-8c72-4b8b-993e-661632e7c056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:50.771259 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1266/2551] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 01:00:50.775930 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c9334-8c72-4b8b-993e-661632e7c056 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:50.777323 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a0ffb-cfee-405f-8acc-34b54f9c6f9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1d6b36254474af1ac33f6e07d8b3942, user_id=fc55962b4ca64a6e9c8f52407da4d763, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:50.779992 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a0ffb-cfee-405f-8acc-34b54f9c6f9d admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:50.785244 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c9334-8c72-4b8b-993e-661632e7c056 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:50.790199 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7de30-6815-407e-b94a-a08efd73cda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:50.790796 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7de30-6815-407e-b94a-a08efd73cda9 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:50.791077 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7de30-6815-407e-b94a-a08efd73cda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:50.791349 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7de30-6815-407e-b94a-a08efd73cda9 admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:50.791986 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d7de30-6815-407e-b94a-a08efd73cda9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 01:00:50.797997 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d7de30-6815-407e-b94a-a08efd73cda9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:50.797997 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d7de30-6815-407e-b94a-a08efd73cda9 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:50.805806 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1267/2552] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 01:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 01:00:50.810179 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7de30-6815-407e-b94a-a08efd73cda9 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:50.816584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:50.820093 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7de30-6815-407e-b94a-a08efd73cda9 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:50.824600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca85f66-137b-4400-bc7d-f32a0fd05ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:50.825163 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca85f66-137b-4400-bc7d-f32a0fd05ac1 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:50.825437 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca85f66-137b-4400-bc7d-f32a0fd05ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:50.825711 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca85f66-137b-4400-bc7d-f32a0fd05ac1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:50.826301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca85f66-137b-4400-bc7d-f32a0fd05ac1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:50.828152 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca85f66-137b-4400-bc7d-f32a0fd05ac1 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:50.934815 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:50.940097 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1286/2553] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:00:50 2022] PUT /identity/v3/projects/f1d6b36254474af1ac33f6e07d8b3942/users/fc55962b4ca64a6e9c8f52407da4d763/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:50.951056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8a7e6-50f6-47e8-bfbe-3263b4f76160 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:50.951340 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8a7e6-50f6-47e8-bfbe-3263b4f76160 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:50.951619 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8a7e6-50f6-47e8-bfbe-3263b4f76160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:50.973223 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1268/2554] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:00:50 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 01:00:50.974205 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca85f66-137b-4400-bc7d-f32a0fd05ac1 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:50.979693 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-a7b8a7e6-50f6-47e8-bfbe-3263b4f76160 None None] MFA Rules not processed for user `fc55962b4ca64a6e9c8f52407da4d763`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:50.980916 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a7b8a7e6-50f6-47e8-bfbe-3263b4f76160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:51.034684 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca85f66-137b-4400-bc7d-f32a0fd05ac1 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:51.039605 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53410b7c-4cf1-47ae-ae0a-22086372a3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:51.040214 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53410b7c-4cf1-47ae-ae0a-22086372a3a3 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:51.040512 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53410b7c-4cf1-47ae-ae0a-22086372a3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:51.040790 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53410b7c-4cf1-47ae-ae0a-22086372a3a3 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:51.041436 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53410b7c-4cf1-47ae-ae0a-22086372a3a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:51.043352 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53410b7c-4cf1-47ae-ae0a-22086372a3a3 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:51.046305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1287/2555] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 01:00:51.058085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8a7e6-50f6-47e8-bfbe-3263b4f76160 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:51.059457 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a7b8a7e6-50f6-47e8-bfbe-3263b4f76160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:51.117158 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8a7e6-50f6-47e8-bfbe-3263b4f76160 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:51.120750 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915ff59-82df-40e7-adc6-da0ea741081b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:51.121361 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915ff59-82df-40e7-adc6-da0ea741081b service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:51.121639 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915ff59-82df-40e7-adc6-da0ea741081b service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:51.121911 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915ff59-82df-40e7-adc6-da0ea741081b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:51.131141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915ff59-82df-40e7-adc6-da0ea741081b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:51.132992 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915ff59-82df-40e7-adc6-da0ea741081b service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:51.137146 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1269/2556] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:00:50 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 01:00:51.138241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53410b7c-4cf1-47ae-ae0a-22086372a3a3 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:51.144378 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1288/2557] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:00:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 01:00:51.148512 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53410b7c-4cf1-47ae-ae0a-22086372a3a3 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:51.153181 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd72666-8ee6-4811-bf7b-e6d1fff2c06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:51.153808 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd72666-8ee6-4811-bf7b-e6d1fff2c06d admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:51.154083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd72666-8ee6-4811-bf7b-e6d1fff2c06d admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:51.154515 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd72666-8ee6-4811-bf7b-e6d1fff2c06d admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:51.155184 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd72666-8ee6-4811-bf7b-e6d1fff2c06d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:51.157551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd72666-8ee6-4811-bf7b-e6d1fff2c06d admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:51.169230 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1270/2558] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 01:00:51.175219 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915ff59-82df-40e7-adc6-da0ea741081b service cinder] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:51.186187 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915ff59-82df-40e7-adc6-da0ea741081b service cinder] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:51.191345 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752ffcd-1aad-499a-a0db-15b221b2dee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:51.191955 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752ffcd-1aad-499a-a0db-15b221b2dee4 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:51.192237 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752ffcd-1aad-499a-a0db-15b221b2dee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:51.192540 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752ffcd-1aad-499a-a0db-15b221b2dee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:51.193187 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752ffcd-1aad-499a-a0db-15b221b2dee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:51.195111 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752ffcd-1aad-499a-a0db-15b221b2dee4 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:51.206221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1289/2559] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:00:51.210646 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd72666-8ee6-4811-bf7b-e6d1fff2c06d admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:51.223402 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd72666-8ee6-4811-bf7b-e6d1fff2c06d admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:51.228154 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1751c7-2884-45cc-8d88-cee4d7723ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:51.228936 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1751c7-2884-45cc-8d88-cee4d7723ab1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:51.229361 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1751c7-2884-45cc-8d88-cee4d7723ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:51.229713 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1751c7-2884-45cc-8d88-cee4d7723ab1 admin admin] PATH_INFO: `/v3/projects/03b021c415a147e08dbfede2ddc8cc27/users/38047508b7d44f05af7116a15ba7e319/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:51.246958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1751c7-2884-45cc-8d88-cee4d7723ab1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03b021c415a147e08dbfede2ddc8cc27, user_id=38047508b7d44f05af7116a15ba7e319, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:51.249671 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1751c7-2884-45cc-8d88-cee4d7723ab1 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:51.322474 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1271/2560] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:00:51 2022] PUT /identity/v3/projects/03b021c415a147e08dbfede2ddc8cc27/users/38047508b7d44f05af7116a15ba7e319/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:51.327522 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14232be5-b62b-4712-b133-7479671f388b None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:51.327812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14232be5-b62b-4712-b133-7479671f388b None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:51.328087 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14232be5-b62b-4712-b133-7479671f388b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:51.347607 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-14232be5-b62b-4712-b133-7479671f388b None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:51.348849 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-14232be5-b62b-4712-b133-7479671f388b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:51.409961 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1290/2561] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:00:51.419594 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1751c7-2884-45cc-8d88-cee4d7723ab1 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:51.421310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-8f1751c7-2884-45cc-8d88-cee4d7723ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:51.480205 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1751c7-2884-45cc-8d88-cee4d7723ab1 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:51.483993 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306a8cc-0457-4b65-9e90-0e45863d87d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:51.484641 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306a8cc-0457-4b65-9e90-0e45863d87d3 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:51.484926 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306a8cc-0457-4b65-9e90-0e45863d87d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:51.485203 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306a8cc-0457-4b65-9e90-0e45863d87d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:51.494548 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f306a8cc-0457-4b65-9e90-0e45863d87d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:51.495970 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f306a8cc-0457-4b65-9e90-0e45863d87d3 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:51.505941 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1272/2562] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:00:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:00:55.391426 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14232be5-b62b-4712-b133-7479671f388b None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:55.405917 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14232be5-b62b-4712-b133-7479671f388b None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:55.418824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42427414-d62e-42e9-a719-a94328592fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:55.420042 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42427414-d62e-42e9-a719-a94328592fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:55.420578 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42427414-d62e-42e9-a719-a94328592fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:55.421184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42427414-d62e-42e9-a719-a94328592fd0 admin admin] PATH_INFO: `/v3/users/fc55962b4ca64a6e9c8f52407da4d763` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:55.429376 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42427414-d62e-42e9-a719-a94328592fd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc55962b4ca64a6e9c8f52407da4d763)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:55.431990 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42427414-d62e-42e9-a719-a94328592fd0 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:55.456226 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:55.651831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-42427414-d62e-42e9-a719-a94328592fd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc55962b4ca64a6e9c8f52407da4d763'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:55.652773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-42427414-d62e-42e9-a719-a94328592fd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc55962b4ca64a6e9c8f52407da4d763'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:55.682011 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-42427414-d62e-42e9-a719-a94328592fd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc55962b4ca64a6e9c8f52407da4d763'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:55.745980 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-42427414-d62e-42e9-a719-a94328592fd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc55962b4ca64a6e9c8f52407da4d763'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:55.746593 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-42427414-d62e-42e9-a719-a94328592fd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc55962b4ca64a6e9c8f52407da4d763'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:55.761757 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1291/2563] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:00:55 2022] DELETE /identity/v3/users/fc55962b4ca64a6e9c8f52407da4d763 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:55.766631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ec7b3-b1e5-4081-974a-f259389f9554 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:55.767094 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ec7b3-b1e5-4081-974a-f259389f9554 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:55.767278 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ec7b3-b1e5-4081-974a-f259389f9554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:55.800720 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-5e6ec7b3-b1e5-4081-974a-f259389f9554 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:55.802110 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-5e6ec7b3-b1e5-4081-974a-f259389f9554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:55.855720 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:55.900623 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1273/2564] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:00:55.909732 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42427414-d62e-42e9-a719-a94328592fd0 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:55.911932 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-42427414-d62e-42e9-a719-a94328592fd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:55.999005 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42427414-d62e-42e9-a719-a94328592fd0 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:56.004731 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4174931-0762-4e4b-a929-95ee36d8ae6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:56.005527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4174931-0762-4e4b-a929-95ee36d8ae6e service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:56.005908 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4174931-0762-4e4b-a929-95ee36d8ae6e service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:56.006511 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4174931-0762-4e4b-a929-95ee36d8ae6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:56.017584 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4174931-0762-4e4b-a929-95ee36d8ae6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:56.019607 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4174931-0762-4e4b-a929-95ee36d8ae6e service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:56.031764 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1292/2565] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:00:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:00:56.552163 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ec7b3-b1e5-4081-974a-f259389f9554 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:56.553910 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-5e6ec7b3-b1e5-4081-974a-f259389f9554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:56.574002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ec7b3-b1e5-4081-974a-f259389f9554 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:56.579083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc94b7d-48dc-4c39-90e3-6ed1284dec16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:56.579941 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc94b7d-48dc-4c39-90e3-6ed1284dec16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:56.580238 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc94b7d-48dc-4c39-90e3-6ed1284dec16 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:56.580543 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc94b7d-48dc-4c39-90e3-6ed1284dec16 admin admin] PATH_INFO: `/v3/projects/f1d6b36254474af1ac33f6e07d8b3942` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:56.581813 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc94b7d-48dc-4c39-90e3-6ed1284dec16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1d6b36254474af1ac33f6e07d8b3942)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:56.583787 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc94b7d-48dc-4c39-90e3-6ed1284dec16 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:56.698149 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2fc94b7d-48dc-4c39-90e3-6ed1284dec16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1d6b36254474af1ac33f6e07d8b3942', 'request_id': 'req-2fc94b7d-48dc-4c39-90e3-6ed1284dec16'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:56.762160 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2fc94b7d-48dc-4c39-90e3-6ed1284dec16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1d6b36254474af1ac33f6e07d8b3942', 'request_id': 'req-2fc94b7d-48dc-4c39-90e3-6ed1284dec16'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:56.770702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2fc94b7d-48dc-4c39-90e3-6ed1284dec16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1d6b36254474af1ac33f6e07d8b3942', 'request_id': 'req-2fc94b7d-48dc-4c39-90e3-6ed1284dec16'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:56.782797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2fc94b7d-48dc-4c39-90e3-6ed1284dec16 admin admin] The token cache is being invalidate because project f1d6b36254474af1ac33f6e07d8b3942 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:00:56.783709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2fc94b7d-48dc-4c39-90e3-6ed1284dec16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:56.784400 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2fc94b7d-48dc-4c39-90e3-6ed1284dec16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:00:56.786132 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1274/2566] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:00:56 2022] DELETE /identity/v3/projects/f1d6b36254474af1ac33f6e07d8b3942 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:56.791758 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb31e4-f471-431c-8576-8c38afdd0c00 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:56.792260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb31e4-f471-431c-8576-8c38afdd0c00 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:56.792260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb31e4-f471-431c-8576-8c38afdd0c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:56.814823 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-bfcb31e4-f471-431c-8576-8c38afdd0c00 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:56.816073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-bfcb31e4-f471-431c-8576-8c38afdd0c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:56.900635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1293/2567] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:00:56.971624 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073b49c-638b-41aa-ad85-b752fa124380 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:56.971915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073b49c-638b-41aa-ad85-b752fa124380 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:56.972198 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073b49c-638b-41aa-ad85-b752fa124380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:56.995242 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-c073b49c-638b-41aa-ad85-b752fa124380 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:56.997170 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-c073b49c-638b-41aa-ad85-b752fa124380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:57.009230 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1275/2568] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:00:57.010701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb31e4-f471-431c-8576-8c38afdd0c00 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:57.024563 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb31e4-f471-431c-8576-8c38afdd0c00 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:57.028160 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70238ea1-24d4-46f1-8443-de47bd9db182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:57.028982 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70238ea1-24d4-46f1-8443-de47bd9db182 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:57.029336 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70238ea1-24d4-46f1-8443-de47bd9db182 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:57.029858 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70238ea1-24d4-46f1-8443-de47bd9db182 admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:57.031185 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70238ea1-24d4-46f1-8443-de47bd9db182 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 01:00:57.031601 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70238ea1-24d4-46f1-8443-de47bd9db182 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:57.033923 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70238ea1-24d4-46f1-8443-de47bd9db182 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:57.046925 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1294/2569] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 01:00:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 01:00:57.051659 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073b49c-638b-41aa-ad85-b752fa124380 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:57.065345 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073b49c-638b-41aa-ad85-b752fa124380 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:57.067180 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d68da-77be-4a72-8023-aeba670812d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:57.067875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d68da-77be-4a72-8023-aeba670812d7 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:57.068162 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d68da-77be-4a72-8023-aeba670812d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:57.068439 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d68da-77be-4a72-8023-aeba670812d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:57.069135 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75d68da-77be-4a72-8023-aeba670812d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:57.071041 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75d68da-77be-4a72-8023-aeba670812d7 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:57.228682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1276/2570] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:00:57 2022] POST /identity/v3/projects => generated 395 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:00:57.231859 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70238ea1-24d4-46f1-8443-de47bd9db182 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:57.244603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70238ea1-24d4-46f1-8443-de47bd9db182 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:57.268275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:57.336300 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62425583-c78c-4df7-af70-e574444babfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:57.338160 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62425583-c78c-4df7-af70-e574444babfe admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:57.338510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62425583-c78c-4df7-af70-e574444babfe admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:57.339451 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62425583-c78c-4df7-af70-e574444babfe admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:57.340314 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62425583-c78c-4df7-af70-e574444babfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:57.344418 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62425583-c78c-4df7-af70-e574444babfe admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:57.443978 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1295/2571] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:00:57 2022] POST /identity/v3/users => generated 298 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 01:00:57.447812 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d68da-77be-4a72-8023-aeba670812d7 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:57.462716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d68da-77be-4a72-8023-aeba670812d7 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:57.470002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2588c-2a13-4f98-a9a7-e1e2c7c7918e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:57.471537 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2588c-2a13-4f98-a9a7-e1e2c7c7918e admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:57.471925 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2588c-2a13-4f98-a9a7-e1e2c7c7918e admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:57.472307 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2588c-2a13-4f98-a9a7-e1e2c7c7918e admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:57.473149 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e2588c-2a13-4f98-a9a7-e1e2c7c7918e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:57.476420 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e2588c-2a13-4f98-a9a7-e1e2c7c7918e admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:57.490499 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1277/2572] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 01:00:57.494046 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62425583-c78c-4df7-af70-e574444babfe admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:57.508347 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62425583-c78c-4df7-af70-e574444babfe admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:57.513444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86487191-e77f-450a-8766-d6f0236a8eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:57.514189 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86487191-e77f-450a-8766-d6f0236a8eaa admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:57.514509 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86487191-e77f-450a-8766-d6f0236a8eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:57.514787 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86487191-e77f-450a-8766-d6f0236a8eaa admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:57.515448 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86487191-e77f-450a-8766-d6f0236a8eaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:57.517101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86487191-e77f-450a-8766-d6f0236a8eaa admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:57.531391 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1296/2573] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 01:00:57.532665 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2588c-2a13-4f98-a9a7-e1e2c7c7918e admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:57.544705 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2588c-2a13-4f98-a9a7-e1e2c7c7918e admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:57.550183 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87331c-0e5f-4392-972d-dbc5509967de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:57.550936 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87331c-0e5f-4392-972d-dbc5509967de admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:57.551336 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87331c-0e5f-4392-972d-dbc5509967de admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:57.551718 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87331c-0e5f-4392-972d-dbc5509967de admin admin] PATH_INFO: `/v3/projects/180a0fbc5dde47ff8dfd650563e67ed4/users/3787cfa0c96a4b45b8dbc5f81ff5de62/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:57.569447 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87331c-0e5f-4392-972d-dbc5509967de admin admin] RBAC: Authorizing `identity:create_grant(project_id=180a0fbc5dde47ff8dfd650563e67ed4, user_id=3787cfa0c96a4b45b8dbc5f81ff5de62, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:57.572323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87331c-0e5f-4392-972d-dbc5509967de admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:57.654080 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1278/2574] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:00:57 2022] PUT /identity/v3/projects/180a0fbc5dde47ff8dfd650563e67ed4/users/3787cfa0c96a4b45b8dbc5f81ff5de62/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:00:57.664124 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde5b5d-24ca-4e08-937e-f7c5575be68b None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:57.664480 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde5b5d-24ca-4e08-937e-f7c5575be68b None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:57.664789 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde5b5d-24ca-4e08-937e-f7c5575be68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:57.696993 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-2dde5b5d-24ca-4e08-937e-f7c5575be68b None None] MFA Rules not processed for user `3787cfa0c96a4b45b8dbc5f81ff5de62`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:57.698615 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-2dde5b5d-24ca-4e08-937e-f7c5575be68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:57.793458 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1297/2575] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:57 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:00:57.805035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87331c-0e5f-4392-972d-dbc5509967de admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:57.807075 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-8c87331c-0e5f-4392-972d-dbc5509967de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:57.874715 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87331c-0e5f-4392-972d-dbc5509967de admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:57.879546 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac79b2-1e7c-43f3-bf2c-4717dae2b86f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:57.880207 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac79b2-1e7c-43f3-bf2c-4717dae2b86f service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:57.880516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac79b2-1e7c-43f3-bf2c-4717dae2b86f service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:57.880863 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac79b2-1e7c-43f3-bf2c-4717dae2b86f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:57.891048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac79b2-1e7c-43f3-bf2c-4717dae2b86f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:57.893404 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac79b2-1e7c-43f3-bf2c-4717dae2b86f service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:57.903858 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1279/2576] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:00:57 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:00:58.861536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5432a97-eedf-4b82-9714-3947fd4af825 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:58.862028 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5432a97-eedf-4b82-9714-3947fd4af825 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:58.862202 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5432a97-eedf-4b82-9714-3947fd4af825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:58.894674 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-d5432a97-eedf-4b82-9714-3947fd4af825 None None] MFA Rules not processed for user `38047508b7d44f05af7116a15ba7e319`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:00:58.895896 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-d5432a97-eedf-4b82-9714-3947fd4af825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:58.967505 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1298/2577] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:00:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:00:58.981983 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac79b2-1e7c-43f3-bf2c-4717dae2b86f service cinder] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:00:58.983621 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-0bac79b2-1e7c-43f3-bf2c-4717dae2b86f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:00:58.989038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:00:59.057447 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac79b2-1e7c-43f3-bf2c-4717dae2b86f service cinder] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:00:59.062928 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd055093-d5a0-451e-aa56-a1d5157c9914 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:00:59.063526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd055093-d5a0-451e-aa56-a1d5157c9914 service nova] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:00:59.063805 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd055093-d5a0-451e-aa56-a1d5157c9914 service nova] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:00:59.064636 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd055093-d5a0-451e-aa56-a1d5157c9914 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:00:59.072417 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd055093-d5a0-451e-aa56-a1d5157c9914 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:00:59.074857 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd055093-d5a0-451e-aa56-a1d5157c9914 service nova] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:00:59.083214 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1280/2578] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:00:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:01:01.178853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5432a97-eedf-4b82-9714-3947fd4af825 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:01.180362 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-d5432a97-eedf-4b82-9714-3947fd4af825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:01.207396 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:01.245868 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5432a97-eedf-4b82-9714-3947fd4af825 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:01.250653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac816b-456c-4c61-9a1f-cf12e98fc428 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:01.251506 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac816b-456c-4c61-9a1f-cf12e98fc428 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:01.251860 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac816b-456c-4c61-9a1f-cf12e98fc428 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:01.252202 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac816b-456c-4c61-9a1f-cf12e98fc428 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:01.254035 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-abac816b-456c-4c61-9a1f-cf12e98fc428 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:01.355458 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac816b-456c-4c61-9a1f-cf12e98fc428 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:01.357027 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac816b-456c-4c61-9a1f-cf12e98fc428 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:01.378526 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1299/2579] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:01:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:01:01.988034 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600dbb37-184d-4d3a-acd3-7a7ecf283299 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:01.988629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600dbb37-184d-4d3a-acd3-7a7ecf283299 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:01.988758 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600dbb37-184d-4d3a-acd3-7a7ecf283299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:02.025565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-600dbb37-184d-4d3a-acd3-7a7ecf283299 None None] MFA Rules not processed for user `38047508b7d44f05af7116a15ba7e319`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:02.027053 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-600dbb37-184d-4d3a-acd3-7a7ecf283299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:02.044584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1281/2580] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 01:01:02.534765 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac816b-456c-4c61-9a1f-cf12e98fc428 service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:02.544892 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac816b-456c-4c61-9a1f-cf12e98fc428 service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:02.548516 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c44d124-43b7-4412-b61c-2dd7603e74df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:02.549124 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44d124-43b7-4412-b61c-2dd7603e74df service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:02.549404 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44d124-43b7-4412-b61c-2dd7603e74df service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:02.549706 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44d124-43b7-4412-b61c-2dd7603e74df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:02.551427 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-9c44d124-43b7-4412-b61c-2dd7603e74df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:02.561244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c44d124-43b7-4412-b61c-2dd7603e74df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:02.562662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c44d124-43b7-4412-b61c-2dd7603e74df service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:02.571908 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1300/2581] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:01:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:01:05.704811 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600dbb37-184d-4d3a-acd3-7a7ecf283299 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:05.706360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-600dbb37-184d-4d3a-acd3-7a7ecf283299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:05.721184 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600dbb37-184d-4d3a-acd3-7a7ecf283299 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:05.725929 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8ba1f-8b5a-4436-9b7f-c0fd923c9583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:05.726639 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8ba1f-8b5a-4436-9b7f-c0fd923c9583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:05.726987 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8ba1f-8b5a-4436-9b7f-c0fd923c9583 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:05.727350 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8ba1f-8b5a-4436-9b7f-c0fd923c9583 admin admin] PATH_INFO: `/v3/users/34240c306dc54f0c8758dbb4b4525376` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:05.728606 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8ba1f-8b5a-4436-9b7f-c0fd923c9583 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34240c306dc54f0c8758dbb4b4525376)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:05.731456 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8ba1f-8b5a-4436-9b7f-c0fd923c9583 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:06.009195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-24f8ba1f-8b5a-4436-9b7f-c0fd923c9583 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34240c306dc54f0c8758dbb4b4525376'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:06.022136 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-24f8ba1f-8b5a-4436-9b7f-c0fd923c9583 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34240c306dc54f0c8758dbb4b4525376'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:06.155951 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-24f8ba1f-8b5a-4436-9b7f-c0fd923c9583 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34240c306dc54f0c8758dbb4b4525376'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:06.156691 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-24f8ba1f-8b5a-4436-9b7f-c0fd923c9583 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34240c306dc54f0c8758dbb4b4525376'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:06.157899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-24f8ba1f-8b5a-4436-9b7f-c0fd923c9583 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34240c306dc54f0c8758dbb4b4525376'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:06.171285 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1282/2582] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:01:05 2022] DELETE /identity/v3/users/34240c306dc54f0c8758dbb4b4525376 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:06.462274 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c44d124-43b7-4412-b61c-2dd7603e74df service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:06.476788 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c44d124-43b7-4412-b61c-2dd7603e74df service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:06.538914 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c09ed-cb45-497c-8ecc-389f046db188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:06.539478 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c09ed-cb45-497c-8ecc-389f046db188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:06.539764 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c09ed-cb45-497c-8ecc-389f046db188 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:06.540051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c09ed-cb45-497c-8ecc-389f046db188 admin admin] PATH_INFO: `/v3/projects/ec39c32378ac4017ad50aa6f930e514b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:06.541229 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6c09ed-cb45-497c-8ecc-389f046db188 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec39c32378ac4017ad50aa6f930e514b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:06.543275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6c09ed-cb45-497c-8ecc-389f046db188 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:06.659211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-fa6c09ed-cb45-497c-8ecc-389f046db188 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec39c32378ac4017ad50aa6f930e514b', 'request_id': 'req-fa6c09ed-cb45-497c-8ecc-389f046db188'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:06.837708 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-fa6c09ed-cb45-497c-8ecc-389f046db188 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec39c32378ac4017ad50aa6f930e514b', 'request_id': 'req-fa6c09ed-cb45-497c-8ecc-389f046db188'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:06.843666 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:06.847421 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-fa6c09ed-cb45-497c-8ecc-389f046db188 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec39c32378ac4017ad50aa6f930e514b', 'request_id': 'req-fa6c09ed-cb45-497c-8ecc-389f046db188'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:06.861291 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-fa6c09ed-cb45-497c-8ecc-389f046db188 admin admin] The token cache is being invalidate because project ec39c32378ac4017ad50aa6f930e514b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:01:06.861648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-fa6c09ed-cb45-497c-8ecc-389f046db188 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:06.862716 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-fa6c09ed-cb45-497c-8ecc-389f046db188 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:06.864498 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1301/2583] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:01:06 2022] DELETE /identity/v3/projects/ec39c32378ac4017ad50aa6f930e514b => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 01:01:06.870043 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631a6e0-043f-429f-9fe1-db869022325a None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:06.870791 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631a6e0-043f-429f-9fe1-db869022325a None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:06.871920 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631a6e0-043f-429f-9fe1-db869022325a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:06.899989 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-d631a6e0-043f-429f-9fe1-db869022325a None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:06.901360 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-d631a6e0-043f-429f-9fe1-db869022325a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:06.954128 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:06.966807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1283/2584] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:06.976707 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d87c12-8d94-44f8-926c-68587a49dfc7 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:06.977002 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d87c12-8d94-44f8-926c-68587a49dfc7 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:06.977282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d87c12-8d94-44f8-926c-68587a49dfc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:06.996625 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-63d87c12-8d94-44f8-926c-68587a49dfc7 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:06.997831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-63d87c12-8d94-44f8-926c-68587a49dfc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:07.006418 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1302/2585] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:07.010595 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631a6e0-043f-429f-9fe1-db869022325a None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:07.020912 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631a6e0-043f-429f-9fe1-db869022325a None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:07.025584 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2cdca-7564-425b-b86f-39f9da2a6bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:07.026146 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2cdca-7564-425b-b86f-39f9da2a6bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:07.026429 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2cdca-7564-425b-b86f-39f9da2a6bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:07.026704 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2cdca-7564-425b-b86f-39f9da2a6bc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:07.027346 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2cdca-7564-425b-b86f-39f9da2a6bc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 01:01:07.027659 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2cdca-7564-425b-b86f-39f9da2a6bc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:07.029276 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2cdca-7564-425b-b86f-39f9da2a6bc4 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:07.041669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1284/2586] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 01:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 01:01:07.045767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d87c12-8d94-44f8-926c-68587a49dfc7 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:07.056597 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d87c12-8d94-44f8-926c-68587a49dfc7 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:07.063302 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b058ad-9b77-480a-9c6d-3c72bdfe8109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:07.063302 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b058ad-9b77-480a-9c6d-3c72bdfe8109 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:07.063302 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b058ad-9b77-480a-9c6d-3c72bdfe8109 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:07.063302 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b058ad-9b77-480a-9c6d-3c72bdfe8109 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:07.064167 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b058ad-9b77-480a-9c6d-3c72bdfe8109 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:07.066145 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b058ad-9b77-480a-9c6d-3c72bdfe8109 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:07.214869 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1303/2587] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:01:07 2022] POST /identity/v3/projects => generated 373 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 01:01:07.218959 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2cdca-7564-425b-b86f-39f9da2a6bc4 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:07.229598 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2cdca-7564-425b-b86f-39f9da2a6bc4 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:07.286065 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546449b0-fae6-42e7-8d82-2c7a83e447a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:07.286685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546449b0-fae6-42e7-8d82-2c7a83e447a8 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:07.286965 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546449b0-fae6-42e7-8d82-2c7a83e447a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:07.287237 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546449b0-fae6-42e7-8d82-2c7a83e447a8 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:07.288158 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546449b0-fae6-42e7-8d82-2c7a83e447a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:07.290174 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546449b0-fae6-42e7-8d82-2c7a83e447a8 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:07.393989 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1285/2588] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:01:07 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:01:07.395294 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b058ad-9b77-480a-9c6d-3c72bdfe8109 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:07.408424 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b058ad-9b77-480a-9c6d-3c72bdfe8109 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:07.415532 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4602e1-4aad-4b1c-b8d2-9cd43d501df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:07.416425 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4602e1-4aad-4b1c-b8d2-9cd43d501df4 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:07.417783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4602e1-4aad-4b1c-b8d2-9cd43d501df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:07.417998 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4602e1-4aad-4b1c-b8d2-9cd43d501df4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:07.418986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4602e1-4aad-4b1c-b8d2-9cd43d501df4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:07.420856 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4602e1-4aad-4b1c-b8d2-9cd43d501df4 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:07.426007 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546449b0-fae6-42e7-8d82-2c7a83e447a8 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:07.428309 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-546449b0-fae6-42e7-8d82-2c7a83e447a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:07.433363 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1304/2589] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:01:07.436715 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4602e1-4aad-4b1c-b8d2-9cd43d501df4 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:07.444042 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546449b0-fae6-42e7-8d82-2c7a83e447a8 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:07.448214 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4602e1-4aad-4b1c-b8d2-9cd43d501df4 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:07.448989 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5ee25-f872-4415-8aac-33969d89cc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:07.449625 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5ee25-f872-4415-8aac-33969d89cc57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:07.449945 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5ee25-f872-4415-8aac-33969d89cc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:07.450219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5ee25-f872-4415-8aac-33969d89cc57 admin admin] PATH_INFO: `/v3/users/3787cfa0c96a4b45b8dbc5f81ff5de62` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:07.452228 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b5ee25-f872-4415-8aac-33969d89cc57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3787cfa0c96a4b45b8dbc5f81ff5de62)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:07.455550 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b5ee25-f872-4415-8aac-33969d89cc57 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:07.456104 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d8c58e-0cc6-41b8-a226-877e2e5c265e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:07.456878 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d8c58e-0cc6-41b8-a226-877e2e5c265e admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:07.457212 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d8c58e-0cc6-41b8-a226-877e2e5c265e admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:07.457568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d8c58e-0cc6-41b8-a226-877e2e5c265e admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:07.458288 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d8c58e-0cc6-41b8-a226-877e2e5c265e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:07.459967 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d8c58e-0cc6-41b8-a226-877e2e5c265e admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:07.475371 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1305/2590] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 01:01:07.492697 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d8c58e-0cc6-41b8-a226-877e2e5c265e admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:07.504744 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d8c58e-0cc6-41b8-a226-877e2e5c265e admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:07.509853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9f513-516e-4bee-b278-f9c8397e54a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:07.510575 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9f513-516e-4bee-b278-f9c8397e54a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:07.510864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9f513-516e-4bee-b278-f9c8397e54a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:07.511155 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9f513-516e-4bee-b278-f9c8397e54a3 admin admin] PATH_INFO: `/v3/projects/e77b9e003c4643898c8c6cf6d90c7184/users/4304cf34878d454892e639874d365560/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:07.527189 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9f513-516e-4bee-b278-f9c8397e54a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e77b9e003c4643898c8c6cf6d90c7184, user_id=4304cf34878d454892e639874d365560, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:07.530026 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9f513-516e-4bee-b278-f9c8397e54a3 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:07.742489 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1306/2591] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:01:07 2022] PUT /identity/v3/projects/e77b9e003c4643898c8c6cf6d90c7184/users/4304cf34878d454892e639874d365560/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 01:01:07.761434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0335431-6237-42d8-8dea-5e1f8f527c7c None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:07.761734 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0335431-6237-42d8-8dea-5e1f8f527c7c None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:07.762035 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0335431-6237-42d8-8dea-5e1f8f527c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:07.793274 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-d0335431-6237-42d8-8dea-5e1f8f527c7c None None] MFA Rules not processed for user `4304cf34878d454892e639874d365560`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:07.793274 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-d0335431-6237-42d8-8dea-5e1f8f527c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:07.828202 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-13b5ee25-f872-4415-8aac-33969d89cc57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3787cfa0c96a4b45b8dbc5f81ff5de62'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:07.840754 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-13b5ee25-f872-4415-8aac-33969d89cc57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3787cfa0c96a4b45b8dbc5f81ff5de62'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:07.862620 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1307/2592] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 01:01:07.874839 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0335431-6237-42d8-8dea-5e1f8f527c7c None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:07.878812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-d0335431-6237-42d8-8dea-5e1f8f527c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:07.953141 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0335431-6237-42d8-8dea-5e1f8f527c7c None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:07.956187 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-13b5ee25-f872-4415-8aac-33969d89cc57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3787cfa0c96a4b45b8dbc5f81ff5de62'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:07.956849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-13b5ee25-f872-4415-8aac-33969d89cc57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3787cfa0c96a4b45b8dbc5f81ff5de62'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:07.957501 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-13b5ee25-f872-4415-8aac-33969d89cc57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3787cfa0c96a4b45b8dbc5f81ff5de62'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:07.959233 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:07.962616 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec137f-152e-4cd6-a30a-612b8a2bad4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:07.963321 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec137f-152e-4cd6-a30a-612b8a2bad4b service nova] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:07.963591 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec137f-152e-4cd6-a30a-612b8a2bad4b service nova] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:07.964149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec137f-152e-4cd6-a30a-612b8a2bad4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:07.965761 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-fbec137f-152e-4cd6-a30a-612b8a2bad4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:07.976142 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1286/2593] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:01:07 2022] DELETE /identity/v3/users/3787cfa0c96a4b45b8dbc5f81ff5de62 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 01:01:07.977667 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec137f-152e-4cd6-a30a-612b8a2bad4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:07.979444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec137f-152e-4cd6-a30a-612b8a2bad4b service nova] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:07.981940 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7b44d-a4e5-4e39-aa7e-6c5d4068da76 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:07.982294 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7b44d-a4e5-4e39-aa7e-6c5d4068da76 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:07.982635 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7b44d-a4e5-4e39-aa7e-6c5d4068da76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:07.999897 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1308/2594] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:01:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 01:01:08.004654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-2fa7b44d-a4e5-4e39-aa7e-6c5d4068da76 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:08.004954 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-2fa7b44d-a4e5-4e39-aa7e-6c5d4068da76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:08.071889 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1287/2595] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:08.080144 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec137f-152e-4cd6-a30a-612b8a2bad4b service nova] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:08.083495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-fbec137f-152e-4cd6-a30a-612b8a2bad4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:08.192269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec137f-152e-4cd6-a30a-612b8a2bad4b service nova] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:08.224303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:08.245246 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77937a09-778e-409a-8b47-fd9dc9c8835a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:08.245863 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77937a09-778e-409a-8b47-fd9dc9c8835a service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:08.246142 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77937a09-778e-409a-8b47-fd9dc9c8835a service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:08.246414 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77937a09-778e-409a-8b47-fd9dc9c8835a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:08.255804 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77937a09-778e-409a-8b47-fd9dc9c8835a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:08.257366 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77937a09-778e-409a-8b47-fd9dc9c8835a service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:08.269842 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1309/2596] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:01:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 01:01:08.757793 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7b44d-a4e5-4e39-aa7e-6c5d4068da76 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:08.758964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-2fa7b44d-a4e5-4e39-aa7e-6c5d4068da76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:08.774535 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7b44d-a4e5-4e39-aa7e-6c5d4068da76 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:08.780639 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d7cb56-caab-4c2c-ac92-9a4f0266ffc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:08.781201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7cb56-caab-4c2c-ac92-9a4f0266ffc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:08.781705 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7cb56-caab-4c2c-ac92-9a4f0266ffc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:08.781992 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7cb56-caab-4c2c-ac92-9a4f0266ffc5 admin admin] PATH_INFO: `/v3/projects/180a0fbc5dde47ff8dfd650563e67ed4` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:08.783752 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7cb56-caab-4c2c-ac92-9a4f0266ffc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=180a0fbc5dde47ff8dfd650563e67ed4)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:08.786306 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7cb56-caab-4c2c-ac92-9a4f0266ffc5 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:08.916378 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f9d7cb56-caab-4c2c-ac92-9a4f0266ffc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '180a0fbc5dde47ff8dfd650563e67ed4', 'request_id': 'req-f9d7cb56-caab-4c2c-ac92-9a4f0266ffc5'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:09.007312 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f9d7cb56-caab-4c2c-ac92-9a4f0266ffc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '180a0fbc5dde47ff8dfd650563e67ed4', 'request_id': 'req-f9d7cb56-caab-4c2c-ac92-9a4f0266ffc5'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:09.018137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f9d7cb56-caab-4c2c-ac92-9a4f0266ffc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '180a0fbc5dde47ff8dfd650563e67ed4', 'request_id': 'req-f9d7cb56-caab-4c2c-ac92-9a4f0266ffc5'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:09.029692 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f9d7cb56-caab-4c2c-ac92-9a4f0266ffc5 admin admin] The token cache is being invalidate because project 180a0fbc5dde47ff8dfd650563e67ed4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:01:09.029984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f9d7cb56-caab-4c2c-ac92-9a4f0266ffc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:09.030627 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-f9d7cb56-caab-4c2c-ac92-9a4f0266ffc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:09.032177 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1288/2597] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:01:08 2022] DELETE /identity/v3/projects/180a0fbc5dde47ff8dfd650563e67ed4 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:11.489606 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77937a09-778e-409a-8b47-fd9dc9c8835a service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:11.489606 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-77937a09-778e-409a-8b47-fd9dc9c8835a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:11.502256 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77937a09-778e-409a-8b47-fd9dc9c8835a service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:11.560860 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae62a8f-deef-4158-b1ee-bcd016f4da9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:11.561511 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae62a8f-deef-4158-b1ee-bcd016f4da9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:11.561795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae62a8f-deef-4158-b1ee-bcd016f4da9e admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:11.562101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae62a8f-deef-4158-b1ee-bcd016f4da9e admin admin] PATH_INFO: `/v3/users/4304cf34878d454892e639874d365560` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:11.563261 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae62a8f-deef-4158-b1ee-bcd016f4da9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4304cf34878d454892e639874d365560)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:11.565760 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae62a8f-deef-4158-b1ee-bcd016f4da9e admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:11.775414 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-4ae62a8f-deef-4158-b1ee-bcd016f4da9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4304cf34878d454892e639874d365560'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:11.776006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-4ae62a8f-deef-4158-b1ee-bcd016f4da9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4304cf34878d454892e639874d365560'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:11.777932 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:11.789422 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-4ae62a8f-deef-4158-b1ee-bcd016f4da9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4304cf34878d454892e639874d365560'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:11.875348 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-4ae62a8f-deef-4158-b1ee-bcd016f4da9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4304cf34878d454892e639874d365560'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:11.876023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-4ae62a8f-deef-4158-b1ee-bcd016f4da9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4304cf34878d454892e639874d365560'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:11.889085 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1310/2598] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:01:11 2022] DELETE /identity/v3/users/4304cf34878d454892e639874d365560 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:11.894108 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42189a-807f-4fb9-9804-4e12fa555892 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:11.894647 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42189a-807f-4fb9-9804-4e12fa555892 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:11.894647 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42189a-807f-4fb9-9804-4e12fa555892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:11.914176 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-9c42189a-807f-4fb9-9804-4e12fa555892 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:11.915368 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-9c42189a-807f-4fb9-9804-4e12fa555892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:11.973244 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1289/2599] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:11.983013 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae62a8f-deef-4158-b1ee-bcd016f4da9e admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:11.984405 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-4ae62a8f-deef-4158-b1ee-bcd016f4da9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:12.041556 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae62a8f-deef-4158-b1ee-bcd016f4da9e admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:12.045000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc5d02-e82c-4d30-a978-b2f041504e65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:12.045600 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc5d02-e82c-4d30-a978-b2f041504e65 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:12.045882 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc5d02-e82c-4d30-a978-b2f041504e65 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:12.046156 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc5d02-e82c-4d30-a978-b2f041504e65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:12.055612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dc5d02-e82c-4d30-a978-b2f041504e65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:12.057092 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dc5d02-e82c-4d30-a978-b2f041504e65 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:12.066950 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1311/2600] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:01:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:01:12.511552 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c42189a-807f-4fb9-9804-4e12fa555892 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:12.513216 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-9c42189a-807f-4fb9-9804-4e12fa555892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:12.527612 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c42189a-807f-4fb9-9804-4e12fa555892 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:12.532290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021ed82-63f1-4c57-8d19-039c5caced99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:12.535346 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021ed82-63f1-4c57-8d19-039c5caced99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:12.535346 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021ed82-63f1-4c57-8d19-039c5caced99 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:12.535346 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021ed82-63f1-4c57-8d19-039c5caced99 admin admin] PATH_INFO: `/v3/projects/e77b9e003c4643898c8c6cf6d90c7184` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:12.535346 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2021ed82-63f1-4c57-8d19-039c5caced99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e77b9e003c4643898c8c6cf6d90c7184)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:12.536645 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2021ed82-63f1-4c57-8d19-039c5caced99 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:12.648698 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2021ed82-63f1-4c57-8d19-039c5caced99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e77b9e003c4643898c8c6cf6d90c7184', 'request_id': 'req-2021ed82-63f1-4c57-8d19-039c5caced99'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:12.749847 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2021ed82-63f1-4c57-8d19-039c5caced99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e77b9e003c4643898c8c6cf6d90c7184', 'request_id': 'req-2021ed82-63f1-4c57-8d19-039c5caced99'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:12.757484 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2021ed82-63f1-4c57-8d19-039c5caced99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e77b9e003c4643898c8c6cf6d90c7184', 'request_id': 'req-2021ed82-63f1-4c57-8d19-039c5caced99'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:12.759070 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:12.774265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2021ed82-63f1-4c57-8d19-039c5caced99 admin admin] The token cache is being invalidate because project e77b9e003c4643898c8c6cf6d90c7184 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:01:12.774672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2021ed82-63f1-4c57-8d19-039c5caced99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:12.775391 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-2021ed82-63f1-4c57-8d19-039c5caced99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:12.776982 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1290/2601] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:01:12 2022] DELETE /identity/v3/projects/e77b9e003c4643898c8c6cf6d90c7184 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:12.781856 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0fa37-2701-4be6-a371-1336bbe06690 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:12.783009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0fa37-2701-4be6-a371-1336bbe06690 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:12.783009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0fa37-2701-4be6-a371-1336bbe06690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:12.803959 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-a4a0fa37-2701-4be6-a371-1336bbe06690 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:12.805786 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a4a0fa37-2701-4be6-a371-1336bbe06690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:12.891544 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1312/2602] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:12.893792 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ce7f7-beb5-4891-b523-f9617bf675ae None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:12.894293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ce7f7-beb5-4891-b523-f9617bf675ae None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:12.895294 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ce7f7-beb5-4891-b523-f9617bf675ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:12.915230 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-6f6ce7f7-beb5-4891-b523-f9617bf675ae None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:12.916485 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-6f6ce7f7-beb5-4891-b523-f9617bf675ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:12.924349 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1291/2603] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:12.928713 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0fa37-2701-4be6-a371-1336bbe06690 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:12.938956 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0fa37-2701-4be6-a371-1336bbe06690 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:12.943489 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10784e-5444-44f3-b6a9-ff79b1b80e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:12.944089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10784e-5444-44f3-b6a9-ff79b1b80e07 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:12.944370 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10784e-5444-44f3-b6a9-ff79b1b80e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:12.944695 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10784e-5444-44f3-b6a9-ff79b1b80e07 admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:12.945339 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c10784e-5444-44f3-b6a9-ff79b1b80e07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 01:01:12.945648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c10784e-5444-44f3-b6a9-ff79b1b80e07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:12.947244 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c10784e-5444-44f3-b6a9-ff79b1b80e07 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:12.963014 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1313/2604] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 01:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 01:01:12.964242 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ce7f7-beb5-4891-b523-f9617bf675ae None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:12.973627 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ce7f7-beb5-4891-b523-f9617bf675ae None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:12.978974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d55026-d36c-4bf9-868e-2e1cd0d84efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:12.979870 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d55026-d36c-4bf9-868e-2e1cd0d84efe admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:12.980148 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d55026-d36c-4bf9-868e-2e1cd0d84efe admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:12.980427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d55026-d36c-4bf9-868e-2e1cd0d84efe admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:12.988131 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d55026-d36c-4bf9-868e-2e1cd0d84efe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:12.988131 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d55026-d36c-4bf9-868e-2e1cd0d84efe admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:13.095652 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1292/2605] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:01:12 2022] POST /identity/v3/projects => generated 387 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 01:01:13.100385 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10784e-5444-44f3-b6a9-ff79b1b80e07 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:13.111822 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10784e-5444-44f3-b6a9-ff79b1b80e07 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:13.175382 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:13.181783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e184cb0-3abc-446c-8879-3348e36e5dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:13.182404 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e184cb0-3abc-446c-8879-3348e36e5dad admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:13.182686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e184cb0-3abc-446c-8879-3348e36e5dad admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:13.182967 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e184cb0-3abc-446c-8879-3348e36e5dad admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:13.183613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e184cb0-3abc-446c-8879-3348e36e5dad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:13.185643 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e184cb0-3abc-446c-8879-3348e36e5dad admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:13.293348 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1314/2606] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:01:13 2022] POST /identity/v3/users => generated 294 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 01:01:13.297710 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d55026-d36c-4bf9-868e-2e1cd0d84efe admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:13.308812 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d55026-d36c-4bf9-868e-2e1cd0d84efe admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:13.313550 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f78e895-3577-431e-ab7c-b9e87db20cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:13.318573 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78e895-3577-431e-ab7c-b9e87db20cd2 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:13.318573 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78e895-3577-431e-ab7c-b9e87db20cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:13.318573 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78e895-3577-431e-ab7c-b9e87db20cd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:13.318573 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f78e895-3577-431e-ab7c-b9e87db20cd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:13.318573 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f78e895-3577-431e-ab7c-b9e87db20cd2 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:13.331395 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1293/2607] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:01:13.332649 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e184cb0-3abc-446c-8879-3348e36e5dad admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:13.342750 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e184cb0-3abc-446c-8879-3348e36e5dad admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:13.347993 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64de7a-7925-4c59-90bb-c508d4474d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:13.348646 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64de7a-7925-4c59-90bb-c508d4474d39 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:13.348930 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64de7a-7925-4c59-90bb-c508d4474d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:13.349233 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64de7a-7925-4c59-90bb-c508d4474d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:13.349889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf64de7a-7925-4c59-90bb-c508d4474d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:13.351562 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf64de7a-7925-4c59-90bb-c508d4474d39 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:13.363830 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1315/2608] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:01:13.368309 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f78e895-3577-431e-ab7c-b9e87db20cd2 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:13.378784 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f78e895-3577-431e-ab7c-b9e87db20cd2 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:13.383545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a996108-73fb-4f42-82af-e7d289e5d80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:13.386932 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a996108-73fb-4f42-82af-e7d289e5d80a admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:13.386932 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a996108-73fb-4f42-82af-e7d289e5d80a admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:13.386932 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a996108-73fb-4f42-82af-e7d289e5d80a admin admin] PATH_INFO: `/v3/projects/fd9117106e2e431ba915b8480a004f09/users/2aeb5feb631944459b1b8aa04bcc5e91/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:13.399962 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a996108-73fb-4f42-82af-e7d289e5d80a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd9117106e2e431ba915b8480a004f09, user_id=2aeb5feb631944459b1b8aa04bcc5e91, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:13.402657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a996108-73fb-4f42-82af-e7d289e5d80a admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:13.469927 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1294/2609] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:01:13 2022] PUT /identity/v3/projects/fd9117106e2e431ba915b8480a004f09/users/2aeb5feb631944459b1b8aa04bcc5e91/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:13.480287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b21ac-cfb0-496d-8f4a-10ab81a907fe None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:13.480617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b21ac-cfb0-496d-8f4a-10ab81a907fe None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:13.480893 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b21ac-cfb0-496d-8f4a-10ab81a907fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:13.508686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-678b21ac-cfb0-496d-8f4a-10ab81a907fe None None] MFA Rules not processed for user `2aeb5feb631944459b1b8aa04bcc5e91`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:13.509892 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-678b21ac-cfb0-496d-8f4a-10ab81a907fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:13.573856 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1316/2610] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:13.578573 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a996108-73fb-4f42-82af-e7d289e5d80a admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:13.639575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a996108-73fb-4f42-82af-e7d289e5d80a admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:13.644196 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f4f10-a77e-4906-972e-db9ec142ff6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:13.644828 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f4f10-a77e-4906-972e-db9ec142ff6e admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:13.645156 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f4f10-a77e-4906-972e-db9ec142ff6e admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:13.645432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f4f10-a77e-4906-972e-db9ec142ff6e admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:13.646050 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f4f10-a77e-4906-972e-db9ec142ff6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:13.647944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f4f10-a77e-4906-972e-db9ec142ff6e admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:13.756382 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1295/2611] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:01:13 2022] POST /identity/v3/projects => generated 387 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:01:13.760366 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b21ac-cfb0-496d-8f4a-10ab81a907fe None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:13.770879 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b21ac-cfb0-496d-8f4a-10ab81a907fe None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:13.827648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2be4c-6e19-4789-92d3-e14dc1b746d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:13.828264 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2be4c-6e19-4789-92d3-e14dc1b746d3 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:13.828557 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2be4c-6e19-4789-92d3-e14dc1b746d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:13.828834 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2be4c-6e19-4789-92d3-e14dc1b746d3 admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:13.830012 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b2be4c-6e19-4789-92d3-e14dc1b746d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:13.831936 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b2be4c-6e19-4789-92d3-e14dc1b746d3 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:13.996434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1317/2612] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:01:13 2022] POST /identity/v3/users => generated 300 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:01:14.012143 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f4f10-a77e-4906-972e-db9ec142ff6e admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:14.012143 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f4f10-a77e-4906-972e-db9ec142ff6e admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:14.015072 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecec266-4299-4f2e-9b00-4d6fa9a30653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:14.015663 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecec266-4299-4f2e-9b00-4d6fa9a30653 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:14.015947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecec266-4299-4f2e-9b00-4d6fa9a30653 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:14.016223 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecec266-4299-4f2e-9b00-4d6fa9a30653 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:14.017396 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecec266-4299-4f2e-9b00-4d6fa9a30653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:14.018994 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecec266-4299-4f2e-9b00-4d6fa9a30653 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:14.029406 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1296/2613] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:01:14.034135 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2be4c-6e19-4789-92d3-e14dc1b746d3 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:14.046689 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2be4c-6e19-4789-92d3-e14dc1b746d3 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:14.049804 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7fb27-e81d-49b6-985d-918a17886154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:14.050458 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7fb27-e81d-49b6-985d-918a17886154 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:14.050742 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7fb27-e81d-49b6-985d-918a17886154 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:14.051017 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7fb27-e81d-49b6-985d-918a17886154 admin admin] PATH_INFO: `/v3/projects/39247c980ded41afbf09cc7314105670/users/126db0f2f1c34d3ca7f225538e3373f7/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:14.067103 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba7fb27-e81d-49b6-985d-918a17886154 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39247c980ded41afbf09cc7314105670, user_id=126db0f2f1c34d3ca7f225538e3373f7, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:14.070024 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba7fb27-e81d-49b6-985d-918a17886154 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:14.178619 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1318/2614] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:01:14 2022] PUT /identity/v3/projects/39247c980ded41afbf09cc7314105670/users/126db0f2f1c34d3ca7f225538e3373f7/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:14.182827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecec266-4299-4f2e-9b00-4d6fa9a30653 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:14.193470 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecec266-4299-4f2e-9b00-4d6fa9a30653 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:14.249029 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3980054-c848-481f-aea9-76ba09dda6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:14.249635 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3980054-c848-481f-aea9-76ba09dda6b7 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:14.249919 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3980054-c848-481f-aea9-76ba09dda6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:14.250202 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3980054-c848-481f-aea9-76ba09dda6b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:14.250878 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3980054-c848-481f-aea9-76ba09dda6b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:14.252490 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3980054-c848-481f-aea9-76ba09dda6b7 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:14.253722 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:14.262704 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1297/2615] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 01:01:14.267337 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7fb27-e81d-49b6-985d-918a17886154 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:14.277774 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7fb27-e81d-49b6-985d-918a17886154 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:14.282901 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd378dc9-952d-4656-98cd-ed0cafda85ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:14.283555 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd378dc9-952d-4656-98cd-ed0cafda85ef admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:14.283838 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd378dc9-952d-4656-98cd-ed0cafda85ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:14.284117 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd378dc9-952d-4656-98cd-ed0cafda85ef admin admin] PATH_INFO: `/v3/projects/39247c980ded41afbf09cc7314105670/users/126db0f2f1c34d3ca7f225538e3373f7/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:14.286087 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd378dc9-952d-4656-98cd-ed0cafda85ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=39247c980ded41afbf09cc7314105670, user_id=126db0f2f1c34d3ca7f225538e3373f7, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:14.289163 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd378dc9-952d-4656-98cd-ed0cafda85ef admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:14.299537 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1319/2616] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:01:14 2022] PUT /identity/v3/projects/39247c980ded41afbf09cc7314105670/users/126db0f2f1c34d3ca7f225538e3373f7/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:14.309887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68d37e-9cd5-49a6-a972-1b136cb7f232 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:14.310206 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68d37e-9cd5-49a6-a972-1b136cb7f232 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:14.310482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68d37e-9cd5-49a6-a972-1b136cb7f232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:14.338128 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-cd68d37e-9cd5-49a6-a972-1b136cb7f232 None None] MFA Rules not processed for user `126db0f2f1c34d3ca7f225538e3373f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:14.339375 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-cd68d37e-9cd5-49a6-a972-1b136cb7f232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:14.407625 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1298/2617] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:14 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:14.420412 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd378dc9-952d-4656-98cd-ed0cafda85ef admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:14.422148 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-dd378dc9-952d-4656-98cd-ed0cafda85ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:14.443327 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:14.479798 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd378dc9-952d-4656-98cd-ed0cafda85ef admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:14.483583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e0a44-c7de-4def-8061-24d37a46a6d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:14.484189 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e0a44-c7de-4def-8061-24d37a46a6d0 service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:14.484486 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e0a44-c7de-4def-8061-24d37a46a6d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:14.484777 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e0a44-c7de-4def-8061-24d37a46a6d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:14.494513 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e0a44-c7de-4def-8061-24d37a46a6d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:14.495981 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e0a44-c7de-4def-8061-24d37a46a6d0 service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:14.505448 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1320/2618] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:01:14 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:01:14.539973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68d37e-9cd5-49a6-a972-1b136cb7f232 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:14.602025 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68d37e-9cd5-49a6-a972-1b136cb7f232 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:14.606585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b6a83-a056-4f7b-b8b9-31a12030966c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:14.607212 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b6a83-a056-4f7b-b8b9-31a12030966c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:14.607491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b6a83-a056-4f7b-b8b9-31a12030966c admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:14.607763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b6a83-a056-4f7b-b8b9-31a12030966c admin admin] PATH_INFO: `/v3/users/2aeb5feb631944459b1b8aa04bcc5e91` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:14.608963 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b6a83-a056-4f7b-b8b9-31a12030966c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aeb5feb631944459b1b8aa04bcc5e91)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:14.611465 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b6a83-a056-4f7b-b8b9-31a12030966c admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:14.826148 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-aa7b6a83-a056-4f7b-b8b9-31a12030966c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aeb5feb631944459b1b8aa04bcc5e91'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:14.838733 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-aa7b6a83-a056-4f7b-b8b9-31a12030966c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aeb5feb631944459b1b8aa04bcc5e91'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:14.932603 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-aa7b6a83-a056-4f7b-b8b9-31a12030966c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aeb5feb631944459b1b8aa04bcc5e91'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:14.933214 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-aa7b6a83-a056-4f7b-b8b9-31a12030966c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aeb5feb631944459b1b8aa04bcc5e91'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:14.933842 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-aa7b6a83-a056-4f7b-b8b9-31a12030966c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aeb5feb631944459b1b8aa04bcc5e91'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:14.943688 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:14.948067 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1299/2619] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:01:14 2022] DELETE /identity/v3/users/2aeb5feb631944459b1b8aa04bcc5e91 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:14.952757 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3bbc3-6600-4113-ab31-4cf1b008417b None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:14.953241 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3bbc3-6600-4113-ab31-4cf1b008417b None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:14.953341 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3bbc3-6600-4113-ab31-4cf1b008417b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:14.973521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-07e3bbc3-6600-4113-ab31-4cf1b008417b None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:14.974739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-07e3bbc3-6600-4113-ab31-4cf1b008417b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:15.035572 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1321/2620] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:15.048641 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b6a83-a056-4f7b-b8b9-31a12030966c admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:15.050089 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-aa7b6a83-a056-4f7b-b8b9-31a12030966c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:15.107902 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b6a83-a056-4f7b-b8b9-31a12030966c admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:15.112016 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ebd08-24bf-4d81-aaf2-b8d97fd70ef8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:15.112654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ebd08-24bf-4d81-aaf2-b8d97fd70ef8 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:15.112938 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ebd08-24bf-4d81-aaf2-b8d97fd70ef8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:15.113206 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ebd08-24bf-4d81-aaf2-b8d97fd70ef8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:15.122803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ebd08-24bf-4d81-aaf2-b8d97fd70ef8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:15.124234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ebd08-24bf-4d81-aaf2-b8d97fd70ef8 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:15.134393 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1300/2621] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:01:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:01:15.677394 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3bbc3-6600-4113-ab31-4cf1b008417b None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:15.679390 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-07e3bbc3-6600-4113-ab31-4cf1b008417b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:15.692105 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3bbc3-6600-4113-ab31-4cf1b008417b None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:15.696769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be8a53-38d0-4d80-b7ba-81d61b3f1990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:15.697404 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be8a53-38d0-4d80-b7ba-81d61b3f1990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:15.697684 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be8a53-38d0-4d80-b7ba-81d61b3f1990 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:15.697952 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be8a53-38d0-4d80-b7ba-81d61b3f1990 admin admin] PATH_INFO: `/v3/projects/fd9117106e2e431ba915b8480a004f09` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:15.699204 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be8a53-38d0-4d80-b7ba-81d61b3f1990 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd9117106e2e431ba915b8480a004f09)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:15.701052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be8a53-38d0-4d80-b7ba-81d61b3f1990 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:15.838311 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f0be8a53-38d0-4d80-b7ba-81d61b3f1990 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd9117106e2e431ba915b8480a004f09', 'request_id': 'req-f0be8a53-38d0-4d80-b7ba-81d61b3f1990'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:15.956073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f0be8a53-38d0-4d80-b7ba-81d61b3f1990 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd9117106e2e431ba915b8480a004f09', 'request_id': 'req-f0be8a53-38d0-4d80-b7ba-81d61b3f1990'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:15.962294 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f0be8a53-38d0-4d80-b7ba-81d61b3f1990 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd9117106e2e431ba915b8480a004f09', 'request_id': 'req-f0be8a53-38d0-4d80-b7ba-81d61b3f1990'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:15.973965 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f0be8a53-38d0-4d80-b7ba-81d61b3f1990 admin admin] The token cache is being invalidate because project fd9117106e2e431ba915b8480a004f09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:01:15.974286 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f0be8a53-38d0-4d80-b7ba-81d61b3f1990 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:15.974956 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-f0be8a53-38d0-4d80-b7ba-81d61b3f1990 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:15.976504 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1322/2622] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:01:15 2022] DELETE /identity/v3/projects/fd9117106e2e431ba915b8480a004f09 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:15.980510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ebd08-24bf-4d81-aaf2-b8d97fd70ef8 service neutron] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:15.990011 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ebd08-24bf-4d81-aaf2-b8d97fd70ef8 service neutron] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:16.046323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c3158-fa6e-4175-8321-4c8e84cc68d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:16.046949 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c3158-fa6e-4175-8321-4c8e84cc68d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:16.048664 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c3158-fa6e-4175-8321-4c8e84cc68d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:16.048664 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c3158-fa6e-4175-8321-4c8e84cc68d2 admin admin] PATH_INFO: `/v3/users/126db0f2f1c34d3ca7f225538e3373f7` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:16.049300 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c3158-fa6e-4175-8321-4c8e84cc68d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=126db0f2f1c34d3ca7f225538e3373f7)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:16.050765 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c3158-fa6e-4175-8321-4c8e84cc68d2 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:16.217089 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-ea3c3158-fa6e-4175-8321-4c8e84cc68d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '126db0f2f1c34d3ca7f225538e3373f7'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:16.231391 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-ea3c3158-fa6e-4175-8321-4c8e84cc68d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '126db0f2f1c34d3ca7f225538e3373f7'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:16.315759 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-ea3c3158-fa6e-4175-8321-4c8e84cc68d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '126db0f2f1c34d3ca7f225538e3373f7'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:16.316486 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-ea3c3158-fa6e-4175-8321-4c8e84cc68d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '126db0f2f1c34d3ca7f225538e3373f7'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:16.317190 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-ea3c3158-fa6e-4175-8321-4c8e84cc68d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '126db0f2f1c34d3ca7f225538e3373f7'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:16.330843 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1301/2623] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:01:15 2022] DELETE /identity/v3/users/126db0f2f1c34d3ca7f225538e3373f7 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:16.859939 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be8a53-38d0-4d80-b7ba-81d61b3f1990 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:16.859939 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f0be8a53-38d0-4d80-b7ba-81d61b3f1990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:16.859939 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be8a53-38d0-4d80-b7ba-81d61b3f1990 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:16.899323 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:16.915618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79475e28-e12d-4786-a4cd-4b0e06050559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:16.916739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79475e28-e12d-4786-a4cd-4b0e06050559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:16.916739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79475e28-e12d-4786-a4cd-4b0e06050559 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:16.917026 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79475e28-e12d-4786-a4cd-4b0e06050559 admin admin] PATH_INFO: `/v3/projects/39247c980ded41afbf09cc7314105670` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:16.917946 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79475e28-e12d-4786-a4cd-4b0e06050559 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39247c980ded41afbf09cc7314105670)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:16.919923 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79475e28-e12d-4786-a4cd-4b0e06050559 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:17.059632 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-79475e28-e12d-4786-a4cd-4b0e06050559 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39247c980ded41afbf09cc7314105670', 'request_id': 'req-79475e28-e12d-4786-a4cd-4b0e06050559'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:17.155759 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-79475e28-e12d-4786-a4cd-4b0e06050559 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39247c980ded41afbf09cc7314105670', 'request_id': 'req-79475e28-e12d-4786-a4cd-4b0e06050559'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:17.162700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-79475e28-e12d-4786-a4cd-4b0e06050559 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39247c980ded41afbf09cc7314105670', 'request_id': 'req-79475e28-e12d-4786-a4cd-4b0e06050559'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:17.175507 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-79475e28-e12d-4786-a4cd-4b0e06050559 admin admin] The token cache is being invalidate because project 39247c980ded41afbf09cc7314105670 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:01:17.175806 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-79475e28-e12d-4786-a4cd-4b0e06050559 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:17.176828 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-79475e28-e12d-4786-a4cd-4b0e06050559 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:17.179739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1323/2624] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:01:16 2022] DELETE /identity/v3/projects/39247c980ded41afbf09cc7314105670 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:17.184582 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a623b-af59-463e-9040-74739d731c43 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:17.185046 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a623b-af59-463e-9040-74739d731c43 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:17.185164 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a623b-af59-463e-9040-74739d731c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:17.205231 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-bb6a623b-af59-463e-9040-74739d731c43 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:17.206543 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-bb6a623b-af59-463e-9040-74739d731c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:17.224537 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:17.267077 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1302/2625] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:17.275088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c672ee-44c9-4d2b-86af-35f2ad5dc959 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:17.275378 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c672ee-44c9-4d2b-86af-35f2ad5dc959 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:17.275686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c672ee-44c9-4d2b-86af-35f2ad5dc959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:17.295860 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-a7c672ee-44c9-4d2b-86af-35f2ad5dc959 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:17.297098 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a7c672ee-44c9-4d2b-86af-35f2ad5dc959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:17.305682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1324/2626] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:17.310221 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a623b-af59-463e-9040-74739d731c43 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:17.320619 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a623b-af59-463e-9040-74739d731c43 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:17.325276 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ddd75f-12f2-41f8-87ef-2487d99782af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:17.325880 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ddd75f-12f2-41f8-87ef-2487d99782af admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:17.326159 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ddd75f-12f2-41f8-87ef-2487d99782af admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:17.326434 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ddd75f-12f2-41f8-87ef-2487d99782af admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:17.327111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ddd75f-12f2-41f8-87ef-2487d99782af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 01:01:17.327413 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ddd75f-12f2-41f8-87ef-2487d99782af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:17.329573 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ddd75f-12f2-41f8-87ef-2487d99782af admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:17.341409 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1303/2627] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 01:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 01:01:17.346445 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c672ee-44c9-4d2b-86af-35f2ad5dc959 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:17.355941 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c672ee-44c9-4d2b-86af-35f2ad5dc959 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:17.361211 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f098131-9259-41e4-9dbd-34175dd266c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:17.361822 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f098131-9259-41e4-9dbd-34175dd266c1 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:17.362107 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f098131-9259-41e4-9dbd-34175dd266c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:17.362382 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f098131-9259-41e4-9dbd-34175dd266c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:17.363114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f098131-9259-41e4-9dbd-34175dd266c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:17.365062 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f098131-9259-41e4-9dbd-34175dd266c1 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:17.482887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1325/2628] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:01:17 2022] POST /identity/v3/projects => generated 393 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:01:17.487480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ddd75f-12f2-41f8-87ef-2487d99782af admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:17.498118 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ddd75f-12f2-41f8-87ef-2487d99782af admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:17.561755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56cf9b-ab47-4ed9-bced-6f24880dfe64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:17.562409 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56cf9b-ab47-4ed9-bced-6f24880dfe64 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:17.562693 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56cf9b-ab47-4ed9-bced-6f24880dfe64 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:17.562971 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56cf9b-ab47-4ed9-bced-6f24880dfe64 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:17.564147 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d56cf9b-ab47-4ed9-bced-6f24880dfe64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:17.566111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d56cf9b-ab47-4ed9-bced-6f24880dfe64 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:17.681541 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1304/2629] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:01:17 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:01:17.685789 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f098131-9259-41e4-9dbd-34175dd266c1 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:17.698236 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f098131-9259-41e4-9dbd-34175dd266c1 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:17.703425 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c05ebf-e374-474a-830d-968be2c97be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:17.704300 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c05ebf-e374-474a-830d-968be2c97be4 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:17.704755 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c05ebf-e374-474a-830d-968be2c97be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:17.705133 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c05ebf-e374-474a-830d-968be2c97be4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:17.705888 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c05ebf-e374-474a-830d-968be2c97be4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:17.707666 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c05ebf-e374-474a-830d-968be2c97be4 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:17.720147 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1326/2630] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 01:01:17.723947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56cf9b-ab47-4ed9-bced-6f24880dfe64 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:17.734342 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56cf9b-ab47-4ed9-bced-6f24880dfe64 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:17.741287 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed44fac-d0ff-4036-af6b-d5e16b923290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:17.741899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed44fac-d0ff-4036-af6b-d5e16b923290 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:17.742230 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed44fac-d0ff-4036-af6b-d5e16b923290 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:17.742502 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed44fac-d0ff-4036-af6b-d5e16b923290 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:17.743144 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed44fac-d0ff-4036-af6b-d5e16b923290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:17.744775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed44fac-d0ff-4036-af6b-d5e16b923290 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:17.757083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1305/2631] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:01:17.761887 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c05ebf-e374-474a-830d-968be2c97be4 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:17.773468 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c05ebf-e374-474a-830d-968be2c97be4 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:17.778709 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347c799-723b-49aa-9658-cea83d84ef10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:17.779388 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347c799-723b-49aa-9658-cea83d84ef10 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:17.779662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347c799-723b-49aa-9658-cea83d84ef10 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:17.779937 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347c799-723b-49aa-9658-cea83d84ef10 admin admin] PATH_INFO: `/v3/projects/f25fd32290c24d668cdcf6736e113675/users/af7622d887dc40e18f435b055bbd980f/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:17.799262 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5347c799-723b-49aa-9658-cea83d84ef10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f25fd32290c24d668cdcf6736e113675, user_id=af7622d887dc40e18f435b055bbd980f, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:17.803159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5347c799-723b-49aa-9658-cea83d84ef10 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:17.871273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1327/2632] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:01:17 2022] PUT /identity/v3/projects/f25fd32290c24d668cdcf6736e113675/users/af7622d887dc40e18f435b055bbd980f/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:17.881575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532eb6ca-1dea-4a20-9d98-29e79a59dab7 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:17.882332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532eb6ca-1dea-4a20-9d98-29e79a59dab7 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:17.883248 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532eb6ca-1dea-4a20-9d98-29e79a59dab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:17.917234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-532eb6ca-1dea-4a20-9d98-29e79a59dab7 None None] MFA Rules not processed for user `af7622d887dc40e18f435b055bbd980f`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:17.918645 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-532eb6ca-1dea-4a20-9d98-29e79a59dab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:17.996926 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347c799-723b-49aa-9658-cea83d84ef10 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:17.999444 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1306/2633] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:18.065495 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347c799-723b-49aa-9658-cea83d84ef10 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:18.070252 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b73dfb-dc85-4572-bb01-aaaa41661831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:18.070932 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b73dfb-dc85-4572-bb01-aaaa41661831 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:18.071204 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b73dfb-dc85-4572-bb01-aaaa41661831 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:18.071457 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b73dfb-dc85-4572-bb01-aaaa41661831 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:18.072100 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b73dfb-dc85-4572-bb01-aaaa41661831 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:18.074070 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b73dfb-dc85-4572-bb01-aaaa41661831 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:18.091283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:18.178418 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1328/2634] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:01:17 2022] POST /identity/v3/projects => generated 391 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:01:18.179912 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532eb6ca-1dea-4a20-9d98-29e79a59dab7 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:18.190878 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532eb6ca-1dea-4a20-9d98-29e79a59dab7 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:18.204068 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:18.253801 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cc5bc-def6-489e-b44e-e7753aff8d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:18.254463 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cc5bc-def6-489e-b44e-e7753aff8d12 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:18.254780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cc5bc-def6-489e-b44e-e7753aff8d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:18.255075 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cc5bc-def6-489e-b44e-e7753aff8d12 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:18.255759 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37cc5bc-def6-489e-b44e-e7753aff8d12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:18.257812 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37cc5bc-def6-489e-b44e-e7753aff8d12 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:18.407802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1307/2635] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:01:18 2022] POST /identity/v3/users => generated 302 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:01:18.412230 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b73dfb-dc85-4572-bb01-aaaa41661831 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:18.423567 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b73dfb-dc85-4572-bb01-aaaa41661831 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:18.428756 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2125da7-8391-4984-aaa8-54a71926af08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:18.428980 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2125da7-8391-4984-aaa8-54a71926af08 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:18.429258 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2125da7-8391-4984-aaa8-54a71926af08 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:18.429525 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2125da7-8391-4984-aaa8-54a71926af08 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:18.430177 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2125da7-8391-4984-aaa8-54a71926af08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:18.431783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2125da7-8391-4984-aaa8-54a71926af08 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:18.444725 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1329/2636] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:01:18.461228 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cc5bc-def6-489e-b44e-e7753aff8d12 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:18.472290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cc5bc-def6-489e-b44e-e7753aff8d12 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:18.477301 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cbf1bb-c7d9-436c-9520-30d48cec42c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:18.478003 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbf1bb-c7d9-436c-9520-30d48cec42c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:18.478291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbf1bb-c7d9-436c-9520-30d48cec42c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:18.478567 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbf1bb-c7d9-436c-9520-30d48cec42c5 admin admin] PATH_INFO: `/v3/projects/47badf187d794b5cab60a2082d4f70a9/users/1912b5d0cfcc4c30a68e008eb8e27e04/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:18.495538 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cbf1bb-c7d9-436c-9520-30d48cec42c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47badf187d794b5cab60a2082d4f70a9, user_id=1912b5d0cfcc4c30a68e008eb8e27e04, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:18.498157 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cbf1bb-c7d9-436c-9520-30d48cec42c5 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:18.715658 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1308/2637] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:01:18 2022] PUT /identity/v3/projects/47badf187d794b5cab60a2082d4f70a9/users/1912b5d0cfcc4c30a68e008eb8e27e04/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:18.720268 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2125da7-8391-4984-aaa8-54a71926af08 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:18.730068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2125da7-8391-4984-aaa8-54a71926af08 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:18.788654 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67b6ce-ea73-49cf-be22-a998b8afc95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:18.789384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67b6ce-ea73-49cf-be22-a998b8afc95b admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:18.789720 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67b6ce-ea73-49cf-be22-a998b8afc95b admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:18.790079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67b6ce-ea73-49cf-be22-a998b8afc95b admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:18.790802 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b67b6ce-ea73-49cf-be22-a998b8afc95b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:18.792497 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b67b6ce-ea73-49cf-be22-a998b8afc95b admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:18.806379 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1330/2638] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:01:18.807513 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cbf1bb-c7d9-436c-9520-30d48cec42c5 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:18.817716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cbf1bb-c7d9-436c-9520-30d48cec42c5 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:18.822184 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad8bde-e3ad-4d16-a924-e86bee8219bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:18.822832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad8bde-e3ad-4d16-a924-e86bee8219bc admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:18.823109 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad8bde-e3ad-4d16-a924-e86bee8219bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:18.823381 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad8bde-e3ad-4d16-a924-e86bee8219bc admin admin] PATH_INFO: `/v3/projects/47badf187d794b5cab60a2082d4f70a9/users/1912b5d0cfcc4c30a68e008eb8e27e04/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:18.825277 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad8bde-e3ad-4d16-a924-e86bee8219bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=47badf187d794b5cab60a2082d4f70a9, user_id=1912b5d0cfcc4c30a68e008eb8e27e04, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:18.827916 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad8bde-e3ad-4d16-a924-e86bee8219bc admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:18.838390 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1309/2639] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:01:18 2022] PUT /identity/v3/projects/47badf187d794b5cab60a2082d4f70a9/users/1912b5d0cfcc4c30a68e008eb8e27e04/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:18.850035 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5975c65-3ef1-4161-ab85-fddc1fe9aa2a None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:18.850326 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5975c65-3ef1-4161-ab85-fddc1fe9aa2a None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:18.850652 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5975c65-3ef1-4161-ab85-fddc1fe9aa2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:18.879904 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-b5975c65-3ef1-4161-ab85-fddc1fe9aa2a None None] MFA Rules not processed for user `1912b5d0cfcc4c30a68e008eb8e27e04`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:18.881168 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-b5975c65-3ef1-4161-ab85-fddc1fe9aa2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:18.950009 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1331/2640] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:18 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:18.960817 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad8bde-e3ad-4d16-a924-e86bee8219bc admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:18.962553 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-c0ad8bde-e3ad-4d16-a924-e86bee8219bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:19.020271 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad8bde-e3ad-4d16-a924-e86bee8219bc admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:19.023853 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74f892-06fb-4ed4-9721-242aafbee6fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:19.024455 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74f892-06fb-4ed4-9721-242aafbee6fd service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:19.024772 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74f892-06fb-4ed4-9721-242aafbee6fd service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:19.025089 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74f892-06fb-4ed4-9721-242aafbee6fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:19.035620 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c74f892-06fb-4ed4-9721-242aafbee6fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:19.037126 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c74f892-06fb-4ed4-9721-242aafbee6fd service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:19.047286 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1310/2641] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:01:18 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:01:19.095511 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5975c65-3ef1-4161-ab85-fddc1fe9aa2a None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:19.096952 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-b5975c65-3ef1-4161-ab85-fddc1fe9aa2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:19.109165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5975c65-3ef1-4161-ab85-fddc1fe9aa2a None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:19.112848 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdfe87e-6552-49bd-a80a-d8c650538b39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:19.113454 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdfe87e-6552-49bd-a80a-d8c650538b39 service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:19.113730 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdfe87e-6552-49bd-a80a-d8c650538b39 service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:19.114003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdfe87e-6552-49bd-a80a-d8c650538b39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:19.171132 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdfe87e-6552-49bd-a80a-d8c650538b39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:19.172676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdfe87e-6552-49bd-a80a-d8c650538b39 service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:19.181741 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1332/2642] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:01:19 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:01:23.872299 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74f892-06fb-4ed4-9721-242aafbee6fd service cinder] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:23.933408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74f892-06fb-4ed4-9721-242aafbee6fd service cinder] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:23.937917 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32faa4-bdb0-4541-83b0-1af5e9c549c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:23.942891 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32faa4-bdb0-4541-83b0-1af5e9c549c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:23.942891 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32faa4-bdb0-4541-83b0-1af5e9c549c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:23.942891 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32faa4-bdb0-4541-83b0-1af5e9c549c8 admin admin] PATH_INFO: `/v3/users/af7622d887dc40e18f435b055bbd980f` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:23.942891 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32faa4-bdb0-4541-83b0-1af5e9c549c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af7622d887dc40e18f435b055bbd980f)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:23.942891 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32faa4-bdb0-4541-83b0-1af5e9c549c8 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:23.956018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:24.164441 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-fc32faa4-bdb0-4541-83b0-1af5e9c549c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af7622d887dc40e18f435b055bbd980f'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:24.176801 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-fc32faa4-bdb0-4541-83b0-1af5e9c549c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af7622d887dc40e18f435b055bbd980f'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:24.251752 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-fc32faa4-bdb0-4541-83b0-1af5e9c549c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af7622d887dc40e18f435b055bbd980f'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:24.252374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-fc32faa4-bdb0-4541-83b0-1af5e9c549c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af7622d887dc40e18f435b055bbd980f'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:24.253015 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-fc32faa4-bdb0-4541-83b0-1af5e9c549c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af7622d887dc40e18f435b055bbd980f'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:24.265304 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1311/2643] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:01:23 2022] DELETE /identity/v3/users/af7622d887dc40e18f435b055bbd980f => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:24.269821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d169406-e795-4286-81d3-fb3b690cfded None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:24.270314 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d169406-e795-4286-81d3-fb3b690cfded None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:24.270415 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d169406-e795-4286-81d3-fb3b690cfded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:24.278252 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:24.290132 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-0d169406-e795-4286-81d3-fb3b690cfded None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:24.291387 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-0d169406-e795-4286-81d3-fb3b690cfded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:24.347788 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1333/2644] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:24.357640 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32faa4-bdb0-4541-83b0-1af5e9c549c8 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:24.359002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-fc32faa4-bdb0-4541-83b0-1af5e9c549c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:24.415114 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32faa4-bdb0-4541-83b0-1af5e9c549c8 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:24.418588 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64876c57-605d-4c95-951b-e63c9ca5ec99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:24.419211 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64876c57-605d-4c95-951b-e63c9ca5ec99 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:24.419471 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64876c57-605d-4c95-951b-e63c9ca5ec99 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:24.419741 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64876c57-605d-4c95-951b-e63c9ca5ec99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:24.429272 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64876c57-605d-4c95-951b-e63c9ca5ec99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:24.430695 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64876c57-605d-4c95-951b-e63c9ca5ec99 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:24.440176 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1312/2645] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:01:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:01:24.877213 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d169406-e795-4286-81d3-fb3b690cfded None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:24.878669 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-0d169406-e795-4286-81d3-fb3b690cfded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:24.893577 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d169406-e795-4286-81d3-fb3b690cfded None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:24.898258 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c7042-822f-429f-bde5-d26b8cbbf049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:24.898898 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c7042-822f-429f-bde5-d26b8cbbf049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:24.899203 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c7042-822f-429f-bde5-d26b8cbbf049 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:24.899475 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c7042-822f-429f-bde5-d26b8cbbf049 admin admin] PATH_INFO: `/v3/projects/f25fd32290c24d668cdcf6736e113675` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:24.900633 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c7042-822f-429f-bde5-d26b8cbbf049 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f25fd32290c24d668cdcf6736e113675)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:24.902580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c7042-822f-429f-bde5-d26b8cbbf049 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:25.019350 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-3c0c7042-822f-429f-bde5-d26b8cbbf049 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f25fd32290c24d668cdcf6736e113675', 'request_id': 'req-3c0c7042-822f-429f-bde5-d26b8cbbf049'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:25.097490 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-3c0c7042-822f-429f-bde5-d26b8cbbf049 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f25fd32290c24d668cdcf6736e113675', 'request_id': 'req-3c0c7042-822f-429f-bde5-d26b8cbbf049'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:25.103634 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-3c0c7042-822f-429f-bde5-d26b8cbbf049 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f25fd32290c24d668cdcf6736e113675', 'request_id': 'req-3c0c7042-822f-429f-bde5-d26b8cbbf049'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:25.115640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-3c0c7042-822f-429f-bde5-d26b8cbbf049 admin admin] The token cache is being invalidate because project f25fd32290c24d668cdcf6736e113675 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:01:25.115934 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-3c0c7042-822f-429f-bde5-d26b8cbbf049 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:25.116616 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-3c0c7042-822f-429f-bde5-d26b8cbbf049 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:25.119088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1334/2646] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:01:24 2022] DELETE /identity/v3/projects/f25fd32290c24d668cdcf6736e113675 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:25.123038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64876c57-605d-4c95-951b-e63c9ca5ec99 service neutron] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:25.132784 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64876c57-605d-4c95-951b-e63c9ca5ec99 service neutron] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:25.187888 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e75812-ea59-434c-a432-8ce1469c2853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:25.188546 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e75812-ea59-434c-a432-8ce1469c2853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:25.188828 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e75812-ea59-434c-a432-8ce1469c2853 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:25.189096 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e75812-ea59-434c-a432-8ce1469c2853 admin admin] PATH_INFO: `/v3/users/1912b5d0cfcc4c30a68e008eb8e27e04` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:25.190202 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e75812-ea59-434c-a432-8ce1469c2853 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1912b5d0cfcc4c30a68e008eb8e27e04)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:25.192111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e75812-ea59-434c-a432-8ce1469c2853 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:25.359962 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:25.368170 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-91e75812-ea59-434c-a432-8ce1469c2853 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1912b5d0cfcc4c30a68e008eb8e27e04'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:25.380021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-91e75812-ea59-434c-a432-8ce1469c2853 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1912b5d0cfcc4c30a68e008eb8e27e04'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:25.429531 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-91e75812-ea59-434c-a432-8ce1469c2853 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1912b5d0cfcc4c30a68e008eb8e27e04'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:25.430168 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-91e75812-ea59-434c-a432-8ce1469c2853 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1912b5d0cfcc4c30a68e008eb8e27e04'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:25.430781 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-91e75812-ea59-434c-a432-8ce1469c2853 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1912b5d0cfcc4c30a68e008eb8e27e04'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:25.444275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1313/2647] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:01:25 2022] DELETE /identity/v3/users/1912b5d0cfcc4c30a68e008eb8e27e04 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:25.876911 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c7042-822f-429f-bde5-d26b8cbbf049 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:25.878366 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-3c0c7042-822f-429f-bde5-d26b8cbbf049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:25.894031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c7042-822f-429f-bde5-d26b8cbbf049 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:25.950961 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd520842-ad27-44cf-b39d-58e5328775ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:25.951598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd520842-ad27-44cf-b39d-58e5328775ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:25.951881 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd520842-ad27-44cf-b39d-58e5328775ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:25.952157 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd520842-ad27-44cf-b39d-58e5328775ff admin admin] PATH_INFO: `/v3/projects/47badf187d794b5cab60a2082d4f70a9` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:25.953335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd520842-ad27-44cf-b39d-58e5328775ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=47badf187d794b5cab60a2082d4f70a9)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:25.955824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd520842-ad27-44cf-b39d-58e5328775ff admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:26.062714 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:26.076076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-cd520842-ad27-44cf-b39d-58e5328775ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47badf187d794b5cab60a2082d4f70a9', 'request_id': 'req-cd520842-ad27-44cf-b39d-58e5328775ff'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:26.131808 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-cd520842-ad27-44cf-b39d-58e5328775ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47badf187d794b5cab60a2082d4f70a9', 'request_id': 'req-cd520842-ad27-44cf-b39d-58e5328775ff'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:26.137471 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-cd520842-ad27-44cf-b39d-58e5328775ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47badf187d794b5cab60a2082d4f70a9', 'request_id': 'req-cd520842-ad27-44cf-b39d-58e5328775ff'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:26.148812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-cd520842-ad27-44cf-b39d-58e5328775ff admin admin] The token cache is being invalidate because project 47badf187d794b5cab60a2082d4f70a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:01:26.149115 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-cd520842-ad27-44cf-b39d-58e5328775ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:26.149728 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-cd520842-ad27-44cf-b39d-58e5328775ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:26.151210 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1335/2648] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:01:25 2022] DELETE /identity/v3/projects/47badf187d794b5cab60a2082d4f70a9 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:26.156198 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214bc08-cdbd-4c18-9241-07a1c474058a None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:26.156736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214bc08-cdbd-4c18-9241-07a1c474058a None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:26.157035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214bc08-cdbd-4c18-9241-07a1c474058a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:26.177179 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-7214bc08-cdbd-4c18-9241-07a1c474058a None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:26.178377 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-7214bc08-cdbd-4c18-9241-07a1c474058a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:26.239176 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1314/2649] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:26.246821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d898a-86b8-46c2-9864-941869dc74f9 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:26.247103 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d898a-86b8-46c2-9864-941869dc74f9 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:26.247398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d898a-86b8-46c2-9864-941869dc74f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:26.267484 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-d47d898a-86b8-46c2-9864-941869dc74f9 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:26.268729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-d47d898a-86b8-46c2-9864-941869dc74f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:26.276726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1336/2650] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:26.280838 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214bc08-cdbd-4c18-9241-07a1c474058a None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:26.290880 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214bc08-cdbd-4c18-9241-07a1c474058a None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:26.295441 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e59298-efa2-41d0-9cb1-320b52960541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:26.296043 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e59298-efa2-41d0-9cb1-320b52960541 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:26.296324 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e59298-efa2-41d0-9cb1-320b52960541 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:26.296641 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e59298-efa2-41d0-9cb1-320b52960541 admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:26.297802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e59298-efa2-41d0-9cb1-320b52960541 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 01:01:26.298107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e59298-efa2-41d0-9cb1-320b52960541 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:26.299697 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e59298-efa2-41d0-9cb1-320b52960541 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:26.311053 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1315/2651] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 01:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 01:01:26.315769 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d898a-86b8-46c2-9864-941869dc74f9 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:26.325139 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d898a-86b8-46c2-9864-941869dc74f9 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:26.330326 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0436a208-79d2-40bc-b054-d6a06ea1c994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:26.330928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436a208-79d2-40bc-b054-d6a06ea1c994 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:26.331210 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436a208-79d2-40bc-b054-d6a06ea1c994 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:26.331485 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436a208-79d2-40bc-b054-d6a06ea1c994 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:26.332118 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0436a208-79d2-40bc-b054-d6a06ea1c994 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:26.334068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0436a208-79d2-40bc-b054-d6a06ea1c994 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:26.495305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1337/2652] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:01:26 2022] POST /identity/v3/projects => generated 389 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 01:01:26.498787 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e59298-efa2-41d0-9cb1-320b52960541 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:26.508665 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e59298-efa2-41d0-9cb1-320b52960541 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:26.563542 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b36b5-f4c6-4a19-a220-a989ab52bf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:26.564145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b36b5-f4c6-4a19-a220-a989ab52bf9a admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:26.564427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b36b5-f4c6-4a19-a220-a989ab52bf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:26.564728 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b36b5-f4c6-4a19-a220-a989ab52bf9a admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:26.565342 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b36b5-f4c6-4a19-a220-a989ab52bf9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:26.567261 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b36b5-f4c6-4a19-a220-a989ab52bf9a admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:26.646216 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1316/2653] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:01:26 2022] POST /identity/v3/users => generated 295 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:01:26.650295 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0436a208-79d2-40bc-b054-d6a06ea1c994 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:26.661043 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0436a208-79d2-40bc-b054-d6a06ea1c994 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:26.667175 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c205f-3fec-4368-a567-1bb3a76761c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:26.667780 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c205f-3fec-4368-a567-1bb3a76761c0 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:26.668061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c205f-3fec-4368-a567-1bb3a76761c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:26.668331 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c205f-3fec-4368-a567-1bb3a76761c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:26.669006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c205f-3fec-4368-a567-1bb3a76761c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:26.670600 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c205f-3fec-4368-a567-1bb3a76761c0 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:26.680996 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1338/2654] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:01:26.685136 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b36b5-f4c6-4a19-a220-a989ab52bf9a admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:26.695538 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b36b5-f4c6-4a19-a220-a989ab52bf9a admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:26.700148 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86356748-1591-4d44-b0c1-c85a18726ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:26.700792 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86356748-1591-4d44-b0c1-c85a18726ae2 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:26.701082 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86356748-1591-4d44-b0c1-c85a18726ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:26.701354 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86356748-1591-4d44-b0c1-c85a18726ae2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:26.702031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86356748-1591-4d44-b0c1-c85a18726ae2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:26.703626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86356748-1591-4d44-b0c1-c85a18726ae2 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:26.714915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1317/2655] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:01:26.719608 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c205f-3fec-4368-a567-1bb3a76761c0 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:26.730399 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c205f-3fec-4368-a567-1bb3a76761c0 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:26.735596 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087a31d-770d-4b05-a8d1-d4a23261d8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:26.736193 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087a31d-770d-4b05-a8d1-d4a23261d8e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:26.736488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087a31d-770d-4b05-a8d1-d4a23261d8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:26.736772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087a31d-770d-4b05-a8d1-d4a23261d8e5 admin admin] PATH_INFO: `/v3/projects/b316f928eef44f8798c2eb43048fea45/users/7eeb8f3b22024cf8ae15ae5c66ed9727/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:26.752552 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087a31d-770d-4b05-a8d1-d4a23261d8e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b316f928eef44f8798c2eb43048fea45, user_id=7eeb8f3b22024cf8ae15ae5c66ed9727, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:26.755266 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087a31d-770d-4b05-a8d1-d4a23261d8e5 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:26.835479 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1339/2656] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:01:26 2022] PUT /identity/v3/projects/b316f928eef44f8798c2eb43048fea45/users/7eeb8f3b22024cf8ae15ae5c66ed9727/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:26.846271 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82052d8-f547-4055-8f2a-9fab2aa91877 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:26.846553 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82052d8-f547-4055-8f2a-9fab2aa91877 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:26.846850 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82052d8-f547-4055-8f2a-9fab2aa91877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:26.875629 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-e82052d8-f547-4055-8f2a-9fab2aa91877 None None] MFA Rules not processed for user `7eeb8f3b22024cf8ae15ae5c66ed9727`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:26.876874 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-e82052d8-f547-4055-8f2a-9fab2aa91877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:26.885394 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:26.941734 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1318/2657] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:26 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:26.950706 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087a31d-770d-4b05-a8d1-d4a23261d8e5 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:27.007712 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087a31d-770d-4b05-a8d1-d4a23261d8e5 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:27.012314 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e8fd4-7af8-43e3-b46b-7320c7f5c274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:27.012950 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e8fd4-7af8-43e3-b46b-7320c7f5c274 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:27.013228 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e8fd4-7af8-43e3-b46b-7320c7f5c274 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:27.013499 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e8fd4-7af8-43e3-b46b-7320c7f5c274 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:27.014124 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e8fd4-7af8-43e3-b46b-7320c7f5c274 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:27.016055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e8fd4-7af8-43e3-b46b-7320c7f5c274 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:27.134691 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1340/2658] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:01:26 2022] POST /identity/v3/projects => generated 387 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:01:27.138674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82052d8-f547-4055-8f2a-9fab2aa91877 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:27.149157 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82052d8-f547-4055-8f2a-9fab2aa91877 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:27.204113 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6dfaf9-e679-4dfe-a6f7-a45ae968127d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:27.204769 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6dfaf9-e679-4dfe-a6f7-a45ae968127d admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:27.204954 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6dfaf9-e679-4dfe-a6f7-a45ae968127d admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:27.205232 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6dfaf9-e679-4dfe-a6f7-a45ae968127d admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:27.205868 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6dfaf9-e679-4dfe-a6f7-a45ae968127d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:27.207800 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6dfaf9-e679-4dfe-a6f7-a45ae968127d admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:27.292050 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1319/2659] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:01:27 2022] POST /identity/v3/users => generated 300 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:01:27.296069 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e8fd4-7af8-43e3-b46b-7320c7f5c274 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:27.307025 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e8fd4-7af8-43e3-b46b-7320c7f5c274 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:27.311544 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0644671-5091-4452-8b35-3e4273703754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:27.312138 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0644671-5091-4452-8b35-3e4273703754 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:27.312444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0644671-5091-4452-8b35-3e4273703754 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:27.312745 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0644671-5091-4452-8b35-3e4273703754 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:27.313400 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0644671-5091-4452-8b35-3e4273703754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:27.317461 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0644671-5091-4452-8b35-3e4273703754 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:27.325305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1341/2660] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:01:27.329484 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6dfaf9-e679-4dfe-a6f7-a45ae968127d admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:27.339416 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6dfaf9-e679-4dfe-a6f7-a45ae968127d admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:27.344557 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d6939-8e12-4305-97e2-820d569aadce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:27.345206 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d6939-8e12-4305-97e2-820d569aadce admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:27.345488 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d6939-8e12-4305-97e2-820d569aadce admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:27.345763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d6939-8e12-4305-97e2-820d569aadce admin admin] PATH_INFO: `/v3/projects/14ed22977f634e6da60037184285dbfa/users/03dc6f2f240f47fe8822371b52ae6851/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:27.360749 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d6939-8e12-4305-97e2-820d569aadce admin admin] RBAC: Authorizing `identity:create_grant(project_id=14ed22977f634e6da60037184285dbfa, user_id=03dc6f2f240f47fe8822371b52ae6851, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:27.363396 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d6939-8e12-4305-97e2-820d569aadce admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:27.458897 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1320/2661] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:01:27 2022] PUT /identity/v3/projects/14ed22977f634e6da60037184285dbfa/users/03dc6f2f240f47fe8822371b52ae6851/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:27.462872 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0644671-5091-4452-8b35-3e4273703754 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:27.473024 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0644671-5091-4452-8b35-3e4273703754 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:27.498513 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:27.529021 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e8987-7d13-438e-ad0d-35bebd1227fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:27.529634 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e8987-7d13-438e-ad0d-35bebd1227fb admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:27.529917 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e8987-7d13-438e-ad0d-35bebd1227fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:27.530191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e8987-7d13-438e-ad0d-35bebd1227fb admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:27.530880 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e8987-7d13-438e-ad0d-35bebd1227fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:27.533555 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e8987-7d13-438e-ad0d-35bebd1227fb admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:27.543634 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1342/2662] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:01:27.548344 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d6939-8e12-4305-97e2-820d569aadce admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:27.558709 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d6939-8e12-4305-97e2-820d569aadce admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:27.563853 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d6e23-6ab5-4d9d-b92f-d4572dbe0677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:27.564524 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d6e23-6ab5-4d9d-b92f-d4572dbe0677 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:27.564808 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d6e23-6ab5-4d9d-b92f-d4572dbe0677 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:27.565085 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d6e23-6ab5-4d9d-b92f-d4572dbe0677 admin admin] PATH_INFO: `/v3/projects/14ed22977f634e6da60037184285dbfa/users/03dc6f2f240f47fe8822371b52ae6851/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:27.566999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d6e23-6ab5-4d9d-b92f-d4572dbe0677 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14ed22977f634e6da60037184285dbfa, user_id=03dc6f2f240f47fe8822371b52ae6851, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:27.569686 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d6e23-6ab5-4d9d-b92f-d4572dbe0677 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:27.579908 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1321/2663] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:01:27 2022] PUT /identity/v3/projects/14ed22977f634e6da60037184285dbfa/users/03dc6f2f240f47fe8822371b52ae6851/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:27.591423 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebbf0f-4bc5-4696-95fd-98a8496d6ea1 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:27.591709 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebbf0f-4bc5-4696-95fd-98a8496d6ea1 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:27.591985 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebbf0f-4bc5-4696-95fd-98a8496d6ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:27.621948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-94ebbf0f-4bc5-4696-95fd-98a8496d6ea1 None None] MFA Rules not processed for user `03dc6f2f240f47fe8822371b52ae6851`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:27.623172 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-94ebbf0f-4bc5-4696-95fd-98a8496d6ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:27.690761 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1343/2664] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:27 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:27.701374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d6e23-6ab5-4d9d-b92f-d4572dbe0677 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:27.703057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-2e4d6e23-6ab5-4d9d-b92f-d4572dbe0677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:27.760556 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d6e23-6ab5-4d9d-b92f-d4572dbe0677 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:27.764115 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400eea2-c837-44ed-b3f0-6cd817e0c836 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:27.764739 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400eea2-c837-44ed-b3f0-6cd817e0c836 service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:27.765021 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400eea2-c837-44ed-b3f0-6cd817e0c836 service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:27.765295 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400eea2-c837-44ed-b3f0-6cd817e0c836 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:27.775798 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6400eea2-c837-44ed-b3f0-6cd817e0c836 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:27.777266 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6400eea2-c837-44ed-b3f0-6cd817e0c836 service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:27.786751 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1322/2665] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:01:27 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 01:01:27.923682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebbf0f-4bc5-4696-95fd-98a8496d6ea1 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:27.984431 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebbf0f-4bc5-4696-95fd-98a8496d6ea1 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:27.989474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7996edb-e694-4f0c-aa52-646340edab8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:27.990098 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7996edb-e694-4f0c-aa52-646340edab8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:27.990377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7996edb-e694-4f0c-aa52-646340edab8e admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:27.990650 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7996edb-e694-4f0c-aa52-646340edab8e admin admin] PATH_INFO: `/v3/users/7eeb8f3b22024cf8ae15ae5c66ed9727` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:27.991753 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7996edb-e694-4f0c-aa52-646340edab8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7eeb8f3b22024cf8ae15ae5c66ed9727)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:27.993679 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7996edb-e694-4f0c-aa52-646340edab8e admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:28.277013 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-e7996edb-e694-4f0c-aa52-646340edab8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7eeb8f3b22024cf8ae15ae5c66ed9727'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:28.277676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-e7996edb-e694-4f0c-aa52-646340edab8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7eeb8f3b22024cf8ae15ae5c66ed9727'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:28.282150 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:28.290438 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-e7996edb-e694-4f0c-aa52-646340edab8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7eeb8f3b22024cf8ae15ae5c66ed9727'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:28.429156 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-e7996edb-e694-4f0c-aa52-646340edab8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7eeb8f3b22024cf8ae15ae5c66ed9727'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:28.429782 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-e7996edb-e694-4f0c-aa52-646340edab8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7eeb8f3b22024cf8ae15ae5c66ed9727'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:28.443510 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1344/2666] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:01:27 2022] DELETE /identity/v3/users/7eeb8f3b22024cf8ae15ae5c66ed9727 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 01:01:28.448316 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c0b57-1e9a-4366-a3f7-56881a625ff2 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:28.448777 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c0b57-1e9a-4366-a3f7-56881a625ff2 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:28.448916 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c0b57-1e9a-4366-a3f7-56881a625ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:28.456901 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:28.467716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-e95c0b57-1e9a-4366-a3f7-56881a625ff2 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:28.469942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-e95c0b57-1e9a-4366-a3f7-56881a625ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:28.526523 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1323/2667] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:28.536572 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7996edb-e694-4f0c-aa52-646340edab8e admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:28.538277 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-e7996edb-e694-4f0c-aa52-646340edab8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:28.595591 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7996edb-e694-4f0c-aa52-646340edab8e admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:28.599240 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3945a4-ca74-4ee2-84f9-f1104f3b9b64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:28.599845 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3945a4-ca74-4ee2-84f9-f1104f3b9b64 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:28.600126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3945a4-ca74-4ee2-84f9-f1104f3b9b64 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:28.600401 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3945a4-ca74-4ee2-84f9-f1104f3b9b64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:28.609274 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3945a4-ca74-4ee2-84f9-f1104f3b9b64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:28.611253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3945a4-ca74-4ee2-84f9-f1104f3b9b64 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:28.620684 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1345/2668] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:01:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:01:29.065718 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c0b57-1e9a-4366-a3f7-56881a625ff2 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:29.067303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-e95c0b57-1e9a-4366-a3f7-56881a625ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:29.079687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c0b57-1e9a-4366-a3f7-56881a625ff2 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:29.084309 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2d519-7c4c-48a8-b5f2-0106a8948f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:29.084978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2d519-7c4c-48a8-b5f2-0106a8948f2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:29.085264 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2d519-7c4c-48a8-b5f2-0106a8948f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:29.085542 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2d519-7c4c-48a8-b5f2-0106a8948f2b admin admin] PATH_INFO: `/v3/projects/b316f928eef44f8798c2eb43048fea45` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:29.086675 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf2d519-7c4c-48a8-b5f2-0106a8948f2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b316f928eef44f8798c2eb43048fea45)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:29.088607 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf2d519-7c4c-48a8-b5f2-0106a8948f2b admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:29.207188 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-fdf2d519-7c4c-48a8-b5f2-0106a8948f2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b316f928eef44f8798c2eb43048fea45', 'request_id': 'req-fdf2d519-7c4c-48a8-b5f2-0106a8948f2b'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:29.297816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-fdf2d519-7c4c-48a8-b5f2-0106a8948f2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b316f928eef44f8798c2eb43048fea45', 'request_id': 'req-fdf2d519-7c4c-48a8-b5f2-0106a8948f2b'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:29.304432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-fdf2d519-7c4c-48a8-b5f2-0106a8948f2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b316f928eef44f8798c2eb43048fea45', 'request_id': 'req-fdf2d519-7c4c-48a8-b5f2-0106a8948f2b'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:29.314669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-fdf2d519-7c4c-48a8-b5f2-0106a8948f2b admin admin] The token cache is being invalidate because project b316f928eef44f8798c2eb43048fea45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:01:29.314959 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-fdf2d519-7c4c-48a8-b5f2-0106a8948f2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:29.315589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-fdf2d519-7c4c-48a8-b5f2-0106a8948f2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:29.317130 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1324/2669] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:01:29 2022] DELETE /identity/v3/projects/b316f928eef44f8798c2eb43048fea45 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:29.321149 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3945a4-ca74-4ee2-84f9-f1104f3b9b64 service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:29.331372 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3945a4-ca74-4ee2-84f9-f1104f3b9b64 service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:29.388018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90a410-dcdd-44e5-8a28-67956f312960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:29.388678 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90a410-dcdd-44e5-8a28-67956f312960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:29.388954 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90a410-dcdd-44e5-8a28-67956f312960 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:29.389227 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90a410-dcdd-44e5-8a28-67956f312960 admin admin] PATH_INFO: `/v3/users/03dc6f2f240f47fe8822371b52ae6851` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:29.390334 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90a410-dcdd-44e5-8a28-67956f312960 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03dc6f2f240f47fe8822371b52ae6851)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:29.392260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90a410-dcdd-44e5-8a28-67956f312960 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:29.573981 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-5a90a410-dcdd-44e5-8a28-67956f312960 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03dc6f2f240f47fe8822371b52ae6851'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:29.574589 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-5a90a410-dcdd-44e5-8a28-67956f312960 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03dc6f2f240f47fe8822371b52ae6851'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:29.586457 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-5a90a410-dcdd-44e5-8a28-67956f312960 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03dc6f2f240f47fe8822371b52ae6851'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:29.646433 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-5a90a410-dcdd-44e5-8a28-67956f312960 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03dc6f2f240f47fe8822371b52ae6851'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:29.647031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-5a90a410-dcdd-44e5-8a28-67956f312960 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03dc6f2f240f47fe8822371b52ae6851'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:29.649168 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:29.660425 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1346/2670] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:01:29 2022] DELETE /identity/v3/users/03dc6f2f240f47fe8822371b52ae6851 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:30.126159 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2d519-7c4c-48a8-b5f2-0106a8948f2b admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:30.127915 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-fdf2d519-7c4c-48a8-b5f2-0106a8948f2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:30.141522 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2d519-7c4c-48a8-b5f2-0106a8948f2b admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:30.199732 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d8150-5a28-44d0-a59f-1884e19eaaa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:30.200361 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d8150-5a28-44d0-a59f-1884e19eaaa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:30.200665 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d8150-5a28-44d0-a59f-1884e19eaaa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:30.200935 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d8150-5a28-44d0-a59f-1884e19eaaa9 admin admin] PATH_INFO: `/v3/projects/14ed22977f634e6da60037184285dbfa` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:30.202035 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d8150-5a28-44d0-a59f-1884e19eaaa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14ed22977f634e6da60037184285dbfa)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:30.203957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d8150-5a28-44d0-a59f-1884e19eaaa9 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:30.303190 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:30.315881 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-6d3d8150-5a28-44d0-a59f-1884e19eaaa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14ed22977f634e6da60037184285dbfa', 'request_id': 'req-6d3d8150-5a28-44d0-a59f-1884e19eaaa9'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:30.400264 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-6d3d8150-5a28-44d0-a59f-1884e19eaaa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14ed22977f634e6da60037184285dbfa', 'request_id': 'req-6d3d8150-5a28-44d0-a59f-1884e19eaaa9'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:30.407583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-6d3d8150-5a28-44d0-a59f-1884e19eaaa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14ed22977f634e6da60037184285dbfa', 'request_id': 'req-6d3d8150-5a28-44d0-a59f-1884e19eaaa9'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:30.418207 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-6d3d8150-5a28-44d0-a59f-1884e19eaaa9 admin admin] The token cache is being invalidate because project 14ed22977f634e6da60037184285dbfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:01:30.418491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-6d3d8150-5a28-44d0-a59f-1884e19eaaa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:30.419129 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-6d3d8150-5a28-44d0-a59f-1884e19eaaa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:30.420631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1325/2671] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:01:30 2022] DELETE /identity/v3/projects/14ed22977f634e6da60037184285dbfa => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:30.425614 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119819c2-166f-4279-9da6-130327cdc6bd None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:30.426093 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119819c2-166f-4279-9da6-130327cdc6bd None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:30.426224 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119819c2-166f-4279-9da6-130327cdc6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:30.447015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-119819c2-166f-4279-9da6-130327cdc6bd None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:30.454335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-119819c2-166f-4279-9da6-130327cdc6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:30.510544 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1347/2672] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:30.519117 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c68435-666b-403f-996a-e1fc3fc7acb9 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:30.519408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c68435-666b-403f-996a-e1fc3fc7acb9 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:30.519689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c68435-666b-403f-996a-e1fc3fc7acb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:30.539068 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-97c68435-666b-403f-996a-e1fc3fc7acb9 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:30.540268 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-97c68435-666b-403f-996a-e1fc3fc7acb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:30.548852 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1326/2673] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:30.553196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119819c2-166f-4279-9da6-130327cdc6bd None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:30.562673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119819c2-166f-4279-9da6-130327cdc6bd None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:30.568253 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13133fba-fe8b-46c6-89aa-7ae1c52d82eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:30.568909 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13133fba-fe8b-46c6-89aa-7ae1c52d82eb admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:30.569186 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13133fba-fe8b-46c6-89aa-7ae1c52d82eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:30.569454 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13133fba-fe8b-46c6-89aa-7ae1c52d82eb admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:30.570101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13133fba-fe8b-46c6-89aa-7ae1c52d82eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 01:01:30.570399 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13133fba-fe8b-46c6-89aa-7ae1c52d82eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:30.572002 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13133fba-fe8b-46c6-89aa-7ae1c52d82eb admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:30.584057 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1348/2674] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 01:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 01:01:30.589512 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c68435-666b-403f-996a-e1fc3fc7acb9 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:30.598622 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c68435-666b-403f-996a-e1fc3fc7acb9 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:30.603186 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7f4d1-a4c5-4489-a471-a23a68d9689f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:30.603775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7f4d1-a4c5-4489-a471-a23a68d9689f admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:30.604055 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7f4d1-a4c5-4489-a471-a23a68d9689f admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:30.604323 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7f4d1-a4c5-4489-a471-a23a68d9689f admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:30.604984 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7f4d1-a4c5-4489-a471-a23a68d9689f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:30.606867 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7f4d1-a4c5-4489-a471-a23a68d9689f admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:30.733718 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1327/2675] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:01:30 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:01:30.738401 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13133fba-fe8b-46c6-89aa-7ae1c52d82eb admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:30.748602 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13133fba-fe8b-46c6-89aa-7ae1c52d82eb admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:30.806784 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e99e29-99e5-418a-acf1-7c727d8b424b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:30.807391 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e99e29-99e5-418a-acf1-7c727d8b424b admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:30.807673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e99e29-99e5-418a-acf1-7c727d8b424b admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:30.807947 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e99e29-99e5-418a-acf1-7c727d8b424b admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:30.808592 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e99e29-99e5-418a-acf1-7c727d8b424b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:30.810622 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e99e29-99e5-418a-acf1-7c727d8b424b admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:30.892671 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1349/2676] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:01:30 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 01:01:30.896803 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7f4d1-a4c5-4489-a471-a23a68d9689f admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:30.908002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7f4d1-a4c5-4489-a471-a23a68d9689f admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:30.912878 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0f0ed-97be-4825-9da2-1d632a38e564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:30.913480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0f0ed-97be-4825-9da2-1d632a38e564 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:30.918707 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0f0ed-97be-4825-9da2-1d632a38e564 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:30.918707 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0f0ed-97be-4825-9da2-1d632a38e564 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:30.918707 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd0f0ed-97be-4825-9da2-1d632a38e564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:30.918707 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd0f0ed-97be-4825-9da2-1d632a38e564 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:30.926766 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1328/2677] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:01:30.930805 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e99e29-99e5-418a-acf1-7c727d8b424b admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:30.940814 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e99e29-99e5-418a-acf1-7c727d8b424b admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:30.945282 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ef9f4-9dd2-4a94-a71e-1e312417a947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:30.947450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ef9f4-9dd2-4a94-a71e-1e312417a947 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:30.947450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ef9f4-9dd2-4a94-a71e-1e312417a947 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:30.947450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ef9f4-9dd2-4a94-a71e-1e312417a947 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:30.947450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80ef9f4-9dd2-4a94-a71e-1e312417a947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:30.948605 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80ef9f4-9dd2-4a94-a71e-1e312417a947 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:30.958965 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1350/2678] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:01:30.963209 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0f0ed-97be-4825-9da2-1d632a38e564 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:30.973860 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0f0ed-97be-4825-9da2-1d632a38e564 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:30.981849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148744a-1b42-4e96-ac5f-4ffce464e4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:30.981849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148744a-1b42-4e96-ac5f-4ffce464e4a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:30.981849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148744a-1b42-4e96-ac5f-4ffce464e4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:30.981849 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148744a-1b42-4e96-ac5f-4ffce464e4a3 admin admin] PATH_INFO: `/v3/projects/0c5dcbd21a704acbae5657f5cc52cf53/users/38e80ff25d5b4656a5ba32301e01cabd/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:30.994494 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9148744a-1b42-4e96-ac5f-4ffce464e4a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c5dcbd21a704acbae5657f5cc52cf53, user_id=38e80ff25d5b4656a5ba32301e01cabd, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:30.997202 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9148744a-1b42-4e96-ac5f-4ffce464e4a3 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:31.120680 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1329/2679] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:01:30 2022] PUT /identity/v3/projects/0c5dcbd21a704acbae5657f5cc52cf53/users/38e80ff25d5b4656a5ba32301e01cabd/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:31.131031 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa46ed-f63b-49ca-988d-9a724d06733f None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:31.131241 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa46ed-f63b-49ca-988d-9a724d06733f None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:31.131606 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa46ed-f63b-49ca-988d-9a724d06733f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:31.159113 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-5afa46ed-f63b-49ca-988d-9a724d06733f None None] MFA Rules not processed for user `38e80ff25d5b4656a5ba32301e01cabd`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:31.160321 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-5afa46ed-f63b-49ca-988d-9a724d06733f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:31.206925 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:31.226886 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1351/2680] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:31.231545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148744a-1b42-4e96-ac5f-4ffce464e4a3 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:31.292846 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148744a-1b42-4e96-ac5f-4ffce464e4a3 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:31.297545 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7eac6b-4409-4f5c-a920-677867eea1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:31.298150 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7eac6b-4409-4f5c-a920-677867eea1b6 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:31.298433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7eac6b-4409-4f5c-a920-677867eea1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:31.298708 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7eac6b-4409-4f5c-a920-677867eea1b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:31.299368 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7eac6b-4409-4f5c-a920-677867eea1b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:31.301293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7eac6b-4409-4f5c-a920-677867eea1b6 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:31.396299 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1330/2681] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:01:31 2022] POST /identity/v3/projects => generated 383 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:01:31.400600 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa46ed-f63b-49ca-988d-9a724d06733f None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:31.411995 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa46ed-f63b-49ca-988d-9a724d06733f None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:31.468106 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14558aac-b6ed-4206-92cd-d0ec024d5748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:31.468766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14558aac-b6ed-4206-92cd-d0ec024d5748 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:31.469644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14558aac-b6ed-4206-92cd-d0ec024d5748 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:31.469815 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14558aac-b6ed-4206-92cd-d0ec024d5748 admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:31.470441 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14558aac-b6ed-4206-92cd-d0ec024d5748 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:31.472369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14558aac-b6ed-4206-92cd-d0ec024d5748 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:31.555335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1352/2682] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:01:31 2022] POST /identity/v3/users => generated 298 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:01:31.559310 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7eac6b-4409-4f5c-a920-677867eea1b6 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:31.570311 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7eac6b-4409-4f5c-a920-677867eea1b6 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:31.574901 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb66a04-d224-469a-a5c6-a11457ba79e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:31.575503 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb66a04-d224-469a-a5c6-a11457ba79e5 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:31.575784 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb66a04-d224-469a-a5c6-a11457ba79e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:31.576058 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb66a04-d224-469a-a5c6-a11457ba79e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:31.576737 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb66a04-d224-469a-a5c6-a11457ba79e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:31.578461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb66a04-d224-469a-a5c6-a11457ba79e5 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:31.588990 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1331/2683] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:01:31.593175 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14558aac-b6ed-4206-92cd-d0ec024d5748 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:31.603220 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14558aac-b6ed-4206-92cd-d0ec024d5748 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:31.607852 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a9b058-2b14-4226-8482-a6457492e286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:31.608730 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9b058-2b14-4226-8482-a6457492e286 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:31.609022 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9b058-2b14-4226-8482-a6457492e286 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:31.609331 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9b058-2b14-4226-8482-a6457492e286 admin admin] PATH_INFO: `/v3/projects/580c1251db644541bd7ead45f1c65861/users/6601c0b3e4b242e9a557a07ffaedd9fc/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:31.624803 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9b058-2b14-4226-8482-a6457492e286 admin admin] RBAC: Authorizing `identity:create_grant(project_id=580c1251db644541bd7ead45f1c65861, user_id=6601c0b3e4b242e9a557a07ffaedd9fc, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:31.628048 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9b058-2b14-4226-8482-a6457492e286 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:31.732248 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1353/2684] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:01:31 2022] PUT /identity/v3/projects/580c1251db644541bd7ead45f1c65861/users/6601c0b3e4b242e9a557a07ffaedd9fc/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:31.736869 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb66a04-d224-469a-a5c6-a11457ba79e5 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:31.747053 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb66a04-d224-469a-a5c6-a11457ba79e5 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:31.772338 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:31.802659 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66cb1fa-dd27-4f23-b18e-1519afa34913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:31.803269 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cb1fa-dd27-4f23-b18e-1519afa34913 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:31.803551 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cb1fa-dd27-4f23-b18e-1519afa34913 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:31.803916 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cb1fa-dd27-4f23-b18e-1519afa34913 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:31.804487 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66cb1fa-dd27-4f23-b18e-1519afa34913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:31.806112 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66cb1fa-dd27-4f23-b18e-1519afa34913 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:31.816719 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1332/2685] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:01:31.824862 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a9b058-2b14-4226-8482-a6457492e286 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:31.835431 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a9b058-2b14-4226-8482-a6457492e286 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:31.840403 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2ee09-ce7e-4191-bb6b-8937132044f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:31.841111 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2ee09-ce7e-4191-bb6b-8937132044f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:31.841395 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2ee09-ce7e-4191-bb6b-8937132044f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:31.841674 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2ee09-ce7e-4191-bb6b-8937132044f7 admin admin] PATH_INFO: `/v3/projects/580c1251db644541bd7ead45f1c65861/users/6601c0b3e4b242e9a557a07ffaedd9fc/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:31.843648 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2ee09-ce7e-4191-bb6b-8937132044f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=580c1251db644541bd7ead45f1c65861, user_id=6601c0b3e4b242e9a557a07ffaedd9fc, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:31.846972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2ee09-ce7e-4191-bb6b-8937132044f7 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:31.860524 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1354/2686] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:01:31 2022] PUT /identity/v3/projects/580c1251db644541bd7ead45f1c65861/users/6601c0b3e4b242e9a557a07ffaedd9fc/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:31.871650 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5596d0e-6d4f-4a21-8847-128a4270f591 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:31.871936 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5596d0e-6d4f-4a21-8847-128a4270f591 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:31.872214 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5596d0e-6d4f-4a21-8847-128a4270f591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:31.914782 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-c5596d0e-6d4f-4a21-8847-128a4270f591 None None] MFA Rules not processed for user `6601c0b3e4b242e9a557a07ffaedd9fc`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:31.915991 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-c5596d0e-6d4f-4a21-8847-128a4270f591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:31.985671 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1333/2687] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:31 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:31.997685 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2ee09-ce7e-4191-bb6b-8937132044f7 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:31.999360 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-b3a2ee09-ce7e-4191-bb6b-8937132044f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:32.057386 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2ee09-ce7e-4191-bb6b-8937132044f7 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:32.060997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3442b-3eb9-48cc-864f-8cd3ff464c9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:32.061601 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3442b-3eb9-48cc-864f-8cd3ff464c9d service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:32.061884 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3442b-3eb9-48cc-864f-8cd3ff464c9d service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:32.062160 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3442b-3eb9-48cc-864f-8cd3ff464c9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:32.071992 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee3442b-3eb9-48cc-864f-8cd3ff464c9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:32.073709 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee3442b-3eb9-48cc-864f-8cd3ff464c9d service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:32.083027 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1355/2688] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:01:31 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:01:32.273491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5596d0e-6d4f-4a21-8847-128a4270f591 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:32.283018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5596d0e-6d4f-4a21-8847-128a4270f591 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:32.286546 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f88ad2-0e7b-44b4-8c6e-6a3f375df76f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:32.287137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f88ad2-0e7b-44b4-8c6e-6a3f375df76f service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:32.287665 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f88ad2-0e7b-44b4-8c6e-6a3f375df76f service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:32.287938 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f88ad2-0e7b-44b4-8c6e-6a3f375df76f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:32.342798 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f88ad2-0e7b-44b4-8c6e-6a3f375df76f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:32.344826 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f88ad2-0e7b-44b4-8c6e-6a3f375df76f service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:32.353476 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1334/2689] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:01:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:01:32.838863 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3442b-3eb9-48cc-864f-8cd3ff464c9d service cinder] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:32.883194 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:32.900819 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3442b-3eb9-48cc-864f-8cd3ff464c9d service cinder] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:32.905605 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded48c40-4594-413a-8333-2fcf5a6d05b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:32.906231 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded48c40-4594-413a-8333-2fcf5a6d05b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:32.907073 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded48c40-4594-413a-8333-2fcf5a6d05b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:32.907396 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded48c40-4594-413a-8333-2fcf5a6d05b5 admin admin] PATH_INFO: `/v3/users/38e80ff25d5b4656a5ba32301e01cabd` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:32.908598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded48c40-4594-413a-8333-2fcf5a6d05b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38e80ff25d5b4656a5ba32301e01cabd)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:32.910519 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded48c40-4594-413a-8333-2fcf5a6d05b5 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:33.191849 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ded48c40-4594-413a-8333-2fcf5a6d05b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38e80ff25d5b4656a5ba32301e01cabd'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:33.192534 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ded48c40-4594-413a-8333-2fcf5a6d05b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38e80ff25d5b4656a5ba32301e01cabd'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:33.204990 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ded48c40-4594-413a-8333-2fcf5a6d05b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38e80ff25d5b4656a5ba32301e01cabd'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:33.298002 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ded48c40-4594-413a-8333-2fcf5a6d05b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38e80ff25d5b4656a5ba32301e01cabd'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:33.298613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ded48c40-4594-413a-8333-2fcf5a6d05b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38e80ff25d5b4656a5ba32301e01cabd'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:33.311536 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1356/2690] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:01:32 2022] DELETE /identity/v3/users/38e80ff25d5b4656a5ba32301e01cabd => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:33.316640 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d06baa-176a-4419-aa05-99acaf19f1ae None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:33.317129 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d06baa-176a-4419-aa05-99acaf19f1ae None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:33.317260 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d06baa-176a-4419-aa05-99acaf19f1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:33.336359 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-03d06baa-176a-4419-aa05-99acaf19f1ae None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:33.337646 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-03d06baa-176a-4419-aa05-99acaf19f1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:33.380261 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:33.395451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1335/2691] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 01:01:33.405305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded48c40-4594-413a-8333-2fcf5a6d05b5 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:33.406741 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-ded48c40-4594-413a-8333-2fcf5a6d05b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:33.463698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded48c40-4594-413a-8333-2fcf5a6d05b5 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:33.467254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84037997-3d75-4f2c-b08d-48d6d56514cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:33.467860 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84037997-3d75-4f2c-b08d-48d6d56514cc service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:33.468140 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84037997-3d75-4f2c-b08d-48d6d56514cc service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:33.468415 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84037997-3d75-4f2c-b08d-48d6d56514cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:33.477759 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84037997-3d75-4f2c-b08d-48d6d56514cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:33.479183 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84037997-3d75-4f2c-b08d-48d6d56514cc service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:33.489265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1357/2692] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:01:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:01:33.883451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d06baa-176a-4419-aa05-99acaf19f1ae None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:33.884996 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-03d06baa-176a-4419-aa05-99acaf19f1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:33.897838 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d06baa-176a-4419-aa05-99acaf19f1ae None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:33.902524 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8418d1-5d87-4a6e-b9a2-4484facf086b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:33.903204 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8418d1-5d87-4a6e-b9a2-4484facf086b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:33.903490 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8418d1-5d87-4a6e-b9a2-4484facf086b admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:33.903786 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8418d1-5d87-4a6e-b9a2-4484facf086b admin admin] PATH_INFO: `/v3/projects/0c5dcbd21a704acbae5657f5cc52cf53` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:33.904999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8418d1-5d87-4a6e-b9a2-4484facf086b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c5dcbd21a704acbae5657f5cc52cf53)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:33.906907 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8418d1-5d87-4a6e-b9a2-4484facf086b admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:34.018195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0d8418d1-5d87-4a6e-b9a2-4484facf086b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c5dcbd21a704acbae5657f5cc52cf53', 'request_id': 'req-0d8418d1-5d87-4a6e-b9a2-4484facf086b'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:34.097379 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0d8418d1-5d87-4a6e-b9a2-4484facf086b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c5dcbd21a704acbae5657f5cc52cf53', 'request_id': 'req-0d8418d1-5d87-4a6e-b9a2-4484facf086b'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:34.103918 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0d8418d1-5d87-4a6e-b9a2-4484facf086b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c5dcbd21a704acbae5657f5cc52cf53', 'request_id': 'req-0d8418d1-5d87-4a6e-b9a2-4484facf086b'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:34.114957 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0d8418d1-5d87-4a6e-b9a2-4484facf086b admin admin] The token cache is being invalidate because project 0c5dcbd21a704acbae5657f5cc52cf53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:01:34.115240 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0d8418d1-5d87-4a6e-b9a2-4484facf086b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:34.115887 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0d8418d1-5d87-4a6e-b9a2-4484facf086b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:34.117398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1336/2693] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:01:33 2022] DELETE /identity/v3/projects/0c5dcbd21a704acbae5657f5cc52cf53 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:34.121374 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84037997-3d75-4f2c-b08d-48d6d56514cc service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:34.130971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84037997-3d75-4f2c-b08d-48d6d56514cc service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:34.188880 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7a33b-1d65-404f-af63-00bd37b43b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:34.189489 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7a33b-1d65-404f-af63-00bd37b43b68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:34.189770 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7a33b-1d65-404f-af63-00bd37b43b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:34.190044 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7a33b-1d65-404f-af63-00bd37b43b68 admin admin] PATH_INFO: `/v3/users/6601c0b3e4b242e9a557a07ffaedd9fc` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:34.191164 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb7a33b-1d65-404f-af63-00bd37b43b68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6601c0b3e4b242e9a557a07ffaedd9fc)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:34.193169 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb7a33b-1d65-404f-af63-00bd37b43b68 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:34.197651 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:34.379947 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2cb7a33b-1d65-404f-af63-00bd37b43b68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6601c0b3e4b242e9a557a07ffaedd9fc'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:34.380645 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2cb7a33b-1d65-404f-af63-00bd37b43b68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6601c0b3e4b242e9a557a07ffaedd9fc'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:34.392922 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2cb7a33b-1d65-404f-af63-00bd37b43b68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6601c0b3e4b242e9a557a07ffaedd9fc'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:34.480532 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2cb7a33b-1d65-404f-af63-00bd37b43b68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6601c0b3e4b242e9a557a07ffaedd9fc'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:34.481229 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2cb7a33b-1d65-404f-af63-00bd37b43b68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6601c0b3e4b242e9a557a07ffaedd9fc'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:34.494212 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1358/2694] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:01:34 2022] DELETE /identity/v3/users/6601c0b3e4b242e9a557a07ffaedd9fc => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:34.911102 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8418d1-5d87-4a6e-b9a2-4484facf086b admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:34.912835 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-0d8418d1-5d87-4a6e-b9a2-4484facf086b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:34.928340 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8418d1-5d87-4a6e-b9a2-4484facf086b admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:34.984396 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee03db-e4d2-4f37-9f12-7eaccb9806a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:34.985615 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee03db-e4d2-4f37-9f12-7eaccb9806a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:34.986001 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee03db-e4d2-4f37-9f12-7eaccb9806a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:34.986307 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee03db-e4d2-4f37-9f12-7eaccb9806a2 admin admin] PATH_INFO: `/v3/projects/580c1251db644541bd7ead45f1c65861` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:34.987474 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee03db-e4d2-4f37-9f12-7eaccb9806a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=580c1251db644541bd7ead45f1c65861)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:34.989421 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee03db-e4d2-4f37-9f12-7eaccb9806a2 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:35.113400 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-caee03db-e4d2-4f37-9f12-7eaccb9806a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '580c1251db644541bd7ead45f1c65861', 'request_id': 'req-caee03db-e4d2-4f37-9f12-7eaccb9806a2'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:35.177095 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-caee03db-e4d2-4f37-9f12-7eaccb9806a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '580c1251db644541bd7ead45f1c65861', 'request_id': 'req-caee03db-e4d2-4f37-9f12-7eaccb9806a2'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:35.183467 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-caee03db-e4d2-4f37-9f12-7eaccb9806a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '580c1251db644541bd7ead45f1c65861', 'request_id': 'req-caee03db-e4d2-4f37-9f12-7eaccb9806a2'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:35.195235 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-caee03db-e4d2-4f37-9f12-7eaccb9806a2 admin admin] The token cache is being invalidate because project 580c1251db644541bd7ead45f1c65861 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:01:35.195526 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-caee03db-e4d2-4f37-9f12-7eaccb9806a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:35.196157 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-caee03db-e4d2-4f37-9f12-7eaccb9806a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:35.197657 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1337/2695] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:01:34 2022] DELETE /identity/v3/projects/580c1251db644541bd7ead45f1c65861 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:35.203481 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750c562-f8cd-416f-8aff-5c8d9653b040 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:35.203972 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750c562-f8cd-416f-8aff-5c8d9653b040 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:35.204076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750c562-f8cd-416f-8aff-5c8d9653b040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:35.223307 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-f750c562-f8cd-416f-8aff-5c8d9653b040 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:35.224540 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f750c562-f8cd-416f-8aff-5c8d9653b040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:35.282792 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1359/2696] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:35.291149 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929485ec-3e33-4671-9495-52d72f35847c None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:35.291438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929485ec-3e33-4671-9495-52d72f35847c None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:35.291715 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929485ec-3e33-4671-9495-52d72f35847c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:35.300007 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:35.312127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-929485ec-3e33-4671-9495-52d72f35847c None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:35.313936 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-929485ec-3e33-4671-9495-52d72f35847c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:35.322073 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1338/2697] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:35.330567 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f750c562-f8cd-416f-8aff-5c8d9653b040 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:35.340436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f750c562-f8cd-416f-8aff-5c8d9653b040 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:35.345662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8eb67-47c3-4d40-bbe8-78e9be54644f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:35.346273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8eb67-47c3-4d40-bbe8-78e9be54644f admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:35.346554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8eb67-47c3-4d40-bbe8-78e9be54644f admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:35.346829 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8eb67-47c3-4d40-bbe8-78e9be54644f admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:35.347475 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac8eb67-47c3-4d40-bbe8-78e9be54644f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 01:01:35.347777 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac8eb67-47c3-4d40-bbe8-78e9be54644f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:35.349411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac8eb67-47c3-4d40-bbe8-78e9be54644f admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:35.361089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1360/2698] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 01:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 01:01:35.365442 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929485ec-3e33-4671-9495-52d72f35847c None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:35.374520 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929485ec-3e33-4671-9495-52d72f35847c None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:35.380123 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32d5e7-94f0-4029-9209-a5492df89d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:35.380737 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32d5e7-94f0-4029-9209-a5492df89d8c admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:35.381017 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32d5e7-94f0-4029-9209-a5492df89d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:35.381284 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32d5e7-94f0-4029-9209-a5492df89d8c admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:35.381898 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32d5e7-94f0-4029-9209-a5492df89d8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:35.383776 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32d5e7-94f0-4029-9209-a5492df89d8c admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:35.486967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1339/2699] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:01:35 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:01:35.491186 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8eb67-47c3-4d40-bbe8-78e9be54644f admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:35.501716 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8eb67-47c3-4d40-bbe8-78e9be54644f admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:35.555978 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650ab77-63fa-4e1b-b280-c2089b2f9feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:35.556601 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650ab77-63fa-4e1b-b280-c2089b2f9feb admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:35.556879 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650ab77-63fa-4e1b-b280-c2089b2f9feb admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:35.557146 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650ab77-63fa-4e1b-b280-c2089b2f9feb admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:35.557768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3650ab77-63fa-4e1b-b280-c2089b2f9feb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:35.559675 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3650ab77-63fa-4e1b-b280-c2089b2f9feb admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:35.647391 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:35.652445 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1361/2700] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:01:35 2022] POST /identity/v3/users => generated 287 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:01:35.656893 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32d5e7-94f0-4029-9209-a5492df89d8c admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:35.668758 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32d5e7-94f0-4029-9209-a5492df89d8c admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:35.673799 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a3aef-2a06-464e-920f-86c74c3a72c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:35.674411 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a3aef-2a06-464e-920f-86c74c3a72c0 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:35.674694 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a3aef-2a06-464e-920f-86c74c3a72c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:35.674969 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a3aef-2a06-464e-920f-86c74c3a72c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:35.675622 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a3aef-2a06-464e-920f-86c74c3a72c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:35.677270 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a3aef-2a06-464e-920f-86c74c3a72c0 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:35.688784 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1340/2701] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:01:35.692238 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650ab77-63fa-4e1b-b280-c2089b2f9feb admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:35.702449 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650ab77-63fa-4e1b-b280-c2089b2f9feb admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:35.707789 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae906e94-4cc3-4f85-b574-26f98de4e3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:35.708562 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae906e94-4cc3-4f85-b574-26f98de4e3ac admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:35.709065 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae906e94-4cc3-4f85-b574-26f98de4e3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:35.709506 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae906e94-4cc3-4f85-b574-26f98de4e3ac admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:35.710500 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae906e94-4cc3-4f85-b574-26f98de4e3ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:35.712161 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae906e94-4cc3-4f85-b574-26f98de4e3ac admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:35.722772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1362/2702] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 01:01:35.726975 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a3aef-2a06-464e-920f-86c74c3a72c0 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:35.736813 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a3aef-2a06-464e-920f-86c74c3a72c0 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:35.741405 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d46827-9db0-48b2-86c9-6617bc0a5eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:35.742044 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d46827-9db0-48b2-86c9-6617bc0a5eee admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:35.742322 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d46827-9db0-48b2-86c9-6617bc0a5eee admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:35.742595 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d46827-9db0-48b2-86c9-6617bc0a5eee admin admin] PATH_INFO: `/v3/projects/c71362698a994b54b6566a48d168fe0e/users/b23fc92d21b7480fad4bcb80563338b0/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:35.757398 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d46827-9db0-48b2-86c9-6617bc0a5eee admin admin] RBAC: Authorizing `identity:create_grant(project_id=c71362698a994b54b6566a48d168fe0e, user_id=b23fc92d21b7480fad4bcb80563338b0, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:35.760024 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d46827-9db0-48b2-86c9-6617bc0a5eee admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:35.858392 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1341/2703] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:01:35 2022] PUT /identity/v3/projects/c71362698a994b54b6566a48d168fe0e/users/b23fc92d21b7480fad4bcb80563338b0/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:35.924835 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02fe9e-d094-4be4-80e6-435aee513189 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:35.925047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02fe9e-d094-4be4-80e6-435aee513189 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:35.925291 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02fe9e-d094-4be4-80e6-435aee513189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:35.953505 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-3b02fe9e-d094-4be4-80e6-435aee513189 None None] MFA Rules not processed for user `b23fc92d21b7480fad4bcb80563338b0`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:35.954749 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-3b02fe9e-d094-4be4-80e6-435aee513189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:36.018573 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1363/2704] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:36.029294 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d46827-9db0-48b2-86c9-6617bc0a5eee admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:36.031049 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-14d46827-9db0-48b2-86c9-6617bc0a5eee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:36.188156 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d46827-9db0-48b2-86c9-6617bc0a5eee admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:36.191793 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb98d00-56bc-4a4c-bf1b-176ed007bbec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:36.192402 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb98d00-56bc-4a4c-bf1b-176ed007bbec service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:36.192722 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb98d00-56bc-4a4c-bf1b-176ed007bbec service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:36.193001 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb98d00-56bc-4a4c-bf1b-176ed007bbec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:36.201355 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb98d00-56bc-4a4c-bf1b-176ed007bbec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:36.202774 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb98d00-56bc-4a4c-bf1b-176ed007bbec service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:36.211952 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1342/2705] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:01:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:01:44.667699 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02fe9e-d094-4be4-80e6-435aee513189 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:44.669242 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-3b02fe9e-d094-4be4-80e6-435aee513189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:44.711323 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02fe9e-d094-4be4-80e6-435aee513189 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:44.715257 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef426d-998d-4b97-8692-47f06baf1e84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:44.715874 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef426d-998d-4b97-8692-47f06baf1e84 service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:44.716156 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef426d-998d-4b97-8692-47f06baf1e84 service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:44.716430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef426d-998d-4b97-8692-47f06baf1e84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:44.718178 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-d1ef426d-998d-4b97-8692-47f06baf1e84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:44.791267 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef426d-998d-4b97-8692-47f06baf1e84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:44.792751 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef426d-998d-4b97-8692-47f06baf1e84 service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:44.802224 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1364/2706] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:01:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:01:46.208097 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=52,keystone:INSERT=2 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:51.185712 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb98d00-56bc-4a4c-bf1b-176ed007bbec service cinder] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:51.287663 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb98d00-56bc-4a4c-bf1b-176ed007bbec service cinder] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:51.292402 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a0e69f-aac2-4e06-8bb4-405a613d8b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:51.293055 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0e69f-aac2-4e06-8bb4-405a613d8b64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:51.293334 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0e69f-aac2-4e06-8bb4-405a613d8b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:51.293605 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0e69f-aac2-4e06-8bb4-405a613d8b64 admin admin] PATH_INFO: `/v3/users/b23fc92d21b7480fad4bcb80563338b0` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:51.294729 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a0e69f-aac2-4e06-8bb4-405a613d8b64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b23fc92d21b7480fad4bcb80563338b0)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:51.296734 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a0e69f-aac2-4e06-8bb4-405a613d8b64 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:51.510148 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-60a0e69f-aac2-4e06-8bb4-405a613d8b64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b23fc92d21b7480fad4bcb80563338b0'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:51.522440 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-60a0e69f-aac2-4e06-8bb4-405a613d8b64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b23fc92d21b7480fad4bcb80563338b0'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:51.582467 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-60a0e69f-aac2-4e06-8bb4-405a613d8b64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b23fc92d21b7480fad4bcb80563338b0'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:51.583103 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-60a0e69f-aac2-4e06-8bb4-405a613d8b64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b23fc92d21b7480fad4bcb80563338b0'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:51.583716 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-60a0e69f-aac2-4e06-8bb4-405a613d8b64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b23fc92d21b7480fad4bcb80563338b0'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:51.595889 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1343/2707] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:01:51 2022] DELETE /identity/v3/users/b23fc92d21b7480fad4bcb80563338b0 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:51.600496 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3142ef0-81f1-468d-839a-99c85b7264c2 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:51.600980 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3142ef0-81f1-468d-839a-99c85b7264c2 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:51.601099 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3142ef0-81f1-468d-839a-99c85b7264c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:51.641017 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-f3142ef0-81f1-468d-839a-99c85b7264c2 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:51.642218 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f3142ef0-81f1-468d-839a-99c85b7264c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:51.653604 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=100 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:51.701729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1365/2708] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:51.711770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a0e69f-aac2-4e06-8bb4-405a613d8b64 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:51.713170 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-60a0e69f-aac2-4e06-8bb4-405a613d8b64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:51.754538 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:51.769855 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a0e69f-aac2-4e06-8bb4-405a613d8b64 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:51.773669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e884151-ef89-4e72-a905-97aace0139db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:51.774271 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e884151-ef89-4e72-a905-97aace0139db service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:51.774553 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e884151-ef89-4e72-a905-97aace0139db service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:51.774826 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e884151-ef89-4e72-a905-97aace0139db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:51.783809 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e884151-ef89-4e72-a905-97aace0139db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:51.785265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e884151-ef89-4e72-a905-97aace0139db service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:51.795332 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1344/2709] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:01:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:01:52.179368 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3142ef0-81f1-468d-839a-99c85b7264c2 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:52.183409 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f3142ef0-81f1-468d-839a-99c85b7264c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:52.194421 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3142ef0-81f1-468d-839a-99c85b7264c2 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:52.200647 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07571f1a-7b0b-4062-8493-f1596f227cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:52.201310 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07571f1a-7b0b-4062-8493-f1596f227cc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:52.201588 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07571f1a-7b0b-4062-8493-f1596f227cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:52.201870 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07571f1a-7b0b-4062-8493-f1596f227cc3 admin admin] PATH_INFO: `/v3/projects/c71362698a994b54b6566a48d168fe0e` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:52.203126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07571f1a-7b0b-4062-8493-f1596f227cc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c71362698a994b54b6566a48d168fe0e)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:52.205098 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07571f1a-7b0b-4062-8493-f1596f227cc3 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:52.314457 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-07571f1a-7b0b-4062-8493-f1596f227cc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c71362698a994b54b6566a48d168fe0e', 'request_id': 'req-07571f1a-7b0b-4062-8493-f1596f227cc3'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:52.371408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-07571f1a-7b0b-4062-8493-f1596f227cc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c71362698a994b54b6566a48d168fe0e', 'request_id': 'req-07571f1a-7b0b-4062-8493-f1596f227cc3'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:52.377973 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-07571f1a-7b0b-4062-8493-f1596f227cc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c71362698a994b54b6566a48d168fe0e', 'request_id': 'req-07571f1a-7b0b-4062-8493-f1596f227cc3'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:52.389662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-07571f1a-7b0b-4062-8493-f1596f227cc3 admin admin] The token cache is being invalidate because project c71362698a994b54b6566a48d168fe0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:01:52.389993 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-07571f1a-7b0b-4062-8493-f1596f227cc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:52.390730 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-07571f1a-7b0b-4062-8493-f1596f227cc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:01:52.392457 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1366/2710] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:01:52 2022] DELETE /identity/v3/projects/c71362698a994b54b6566a48d168fe0e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:52.397273 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395e0f2-159f-4e7d-ba0b-5c6828e08b03 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:52.397731 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395e0f2-159f-4e7d-ba0b-5c6828e08b03 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:52.397831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395e0f2-159f-4e7d-ba0b-5c6828e08b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:52.452707 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-0395e0f2-159f-4e7d-ba0b-5c6828e08b03 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:52.453911 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-0395e0f2-159f-4e7d-ba0b-5c6828e08b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:52.511604 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1345/2711] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:52.519603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41cc90-f007-41b6-9730-0f2aec76429a None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:52.519896 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41cc90-f007-41b6-9730-0f2aec76429a None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:52.520189 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41cc90-f007-41b6-9730-0f2aec76429a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:52.540232 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-2f41cc90-f007-41b6-9730-0f2aec76429a None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:52.541475 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-2f41cc90-f007-41b6-9730-0f2aec76429a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:52.550107 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1367/2712] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:52.554206 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395e0f2-159f-4e7d-ba0b-5c6828e08b03 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:52.563685 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395e0f2-159f-4e7d-ba0b-5c6828e08b03 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:52.568293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81206e3e-d163-40a6-b27c-e95f8f37f090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:52.568870 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81206e3e-d163-40a6-b27c-e95f8f37f090 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:52.569153 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81206e3e-d163-40a6-b27c-e95f8f37f090 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:52.569426 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81206e3e-d163-40a6-b27c-e95f8f37f090 admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:52.570067 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81206e3e-d163-40a6-b27c-e95f8f37f090 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 01:01:52.570369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81206e3e-d163-40a6-b27c-e95f8f37f090 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:52.571956 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81206e3e-d163-40a6-b27c-e95f8f37f090 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:52.583142 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1346/2713] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 01:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 01:01:52.587217 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41cc90-f007-41b6-9730-0f2aec76429a None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:52.596704 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41cc90-f007-41b6-9730-0f2aec76429a None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:52.601260 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf2edc-39cc-4e78-9e37-7b9ee2eb2db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:52.601839 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf2edc-39cc-4e78-9e37-7b9ee2eb2db8 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:52.602120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf2edc-39cc-4e78-9e37-7b9ee2eb2db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:52.602389 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf2edc-39cc-4e78-9e37-7b9ee2eb2db8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:52.603003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf2edc-39cc-4e78-9e37-7b9ee2eb2db8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:52.604918 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf2edc-39cc-4e78-9e37-7b9ee2eb2db8 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:52.721635 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1368/2714] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:01:52 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 01:01:52.725986 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81206e3e-d163-40a6-b27c-e95f8f37f090 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:52.736174 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81206e3e-d163-40a6-b27c-e95f8f37f090 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:52.793388 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf8761-7b4a-4cd6-b52a-40d299c72e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:52.793993 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf8761-7b4a-4cd6-b52a-40d299c72e24 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:52.794276 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf8761-7b4a-4cd6-b52a-40d299c72e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:52.794554 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf8761-7b4a-4cd6-b52a-40d299c72e24 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:52.795172 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedf8761-7b4a-4cd6-b52a-40d299c72e24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:52.797163 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedf8761-7b4a-4cd6-b52a-40d299c72e24 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:52.887530 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1347/2715] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:01:52 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 01:01:52.890591 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf2edc-39cc-4e78-9e37-7b9ee2eb2db8 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:52.901034 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf2edc-39cc-4e78-9e37-7b9ee2eb2db8 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:52.906274 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14634060-a24b-42b5-84fb-105c8b76747a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:52.906877 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14634060-a24b-42b5-84fb-105c8b76747a admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:52.907208 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14634060-a24b-42b5-84fb-105c8b76747a admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:52.907474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14634060-a24b-42b5-84fb-105c8b76747a admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:52.908133 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14634060-a24b-42b5-84fb-105c8b76747a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:52.909792 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14634060-a24b-42b5-84fb-105c8b76747a admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:52.919896 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1369/2716] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 01:01:52.923782 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf8761-7b4a-4cd6-b52a-40d299c72e24 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:52.933822 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf8761-7b4a-4cd6-b52a-40d299c72e24 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:52.938480 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0f625-a270-49c0-b017-e758e151edfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:52.939082 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0f625-a270-49c0-b017-e758e151edfa admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:52.939366 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0f625-a270-49c0-b017-e758e151edfa admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:52.939640 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0f625-a270-49c0-b017-e758e151edfa admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:52.940277 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d0f625-a270-49c0-b017-e758e151edfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:52.941902 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d0f625-a270-49c0-b017-e758e151edfa admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:52.953241 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1348/2717] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:01:52.957372 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14634060-a24b-42b5-84fb-105c8b76747a admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:52.967646 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14634060-a24b-42b5-84fb-105c8b76747a admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:52.972937 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c72ef2-eb2a-4535-8d65-490d13a9484a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:52.975768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c72ef2-eb2a-4535-8d65-490d13a9484a admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:52.975768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c72ef2-eb2a-4535-8d65-490d13a9484a admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:52.975768 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c72ef2-eb2a-4535-8d65-490d13a9484a admin admin] PATH_INFO: `/v3/projects/a223be1c2f6a4cd281b20b0bc7281554/users/83076e7b11fb446ca50f23fe8e6e7ab6/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:52.990406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c72ef2-eb2a-4535-8d65-490d13a9484a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a223be1c2f6a4cd281b20b0bc7281554, user_id=83076e7b11fb446ca50f23fe8e6e7ab6, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:52.993379 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c72ef2-eb2a-4535-8d65-490d13a9484a admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:53.132287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1370/2718] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:01:52 2022] PUT /identity/v3/projects/a223be1c2f6a4cd281b20b0bc7281554/users/83076e7b11fb446ca50f23fe8e6e7ab6/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:01:53.137427 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9b090-87de-42fe-be7f-39fecb889af2 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:53.137767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9b090-87de-42fe-be7f-39fecb889af2 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:53.137995 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9b090-87de-42fe-be7f-39fecb889af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:53.157456 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-a3c9b090-87de-42fe-be7f-39fecb889af2 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:53.158647 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-a3c9b090-87de-42fe-be7f-39fecb889af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:53.175439 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:53.218693 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1349/2719] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 01:01:53.230606 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c72ef2-eb2a-4535-8d65-490d13a9484a admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:53.232501 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a3c72ef2-eb2a-4535-8d65-490d13a9484a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:53.278924 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:01:53.295688 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c72ef2-eb2a-4535-8d65-490d13a9484a admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:53.299977 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb2452c-7485-4d1b-9367-dab6181e7dd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:53.302133 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2452c-7485-4d1b-9367-dab6181e7dd8 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:53.302133 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2452c-7485-4d1b-9367-dab6181e7dd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:53.302133 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2452c-7485-4d1b-9367-dab6181e7dd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:53.310481 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb2452c-7485-4d1b-9367-dab6181e7dd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:53.312058 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb2452c-7485-4d1b-9367-dab6181e7dd8 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:53.322591 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1371/2720] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:01:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:01:53.523150 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c17ca1-e4e4-4894-bc5c-f0091ab5039f None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:53.523726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c17ca1-e4e4-4894-bc5c-f0091ab5039f None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:53.523726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c17ca1-e4e4-4894-bc5c-f0091ab5039f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:53.543565 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-70c17ca1-e4e4-4894-bc5c-f0091ab5039f None None] MFA Rules not processed for user `cd45f1813efc4926a30454aaa108ac47`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:01:53.544832 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-70c17ca1-e4e4-4894-bc5c-f0091ab5039f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:53.615074 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1350/2721] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:01:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:01:55.513391 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb2452c-7485-4d1b-9367-dab6181e7dd8 service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:01:55.515529 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-8cb2452c-7485-4d1b-9367-dab6181e7dd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:55.649106 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb2452c-7485-4d1b-9367-dab6181e7dd8 service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:01:55.654325 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bf32d-2037-4900-8c4d-96be7e4153da service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2876f4182f0949039b862f053533fe85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:01:55.655098 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bf32d-2037-4900-8c4d-96be7e4153da service placement] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:01:55.655933 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bf32d-2037-4900-8c4d-96be7e4153da service placement] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:01:55.656248 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bf32d-2037-4900-8c4d-96be7e4153da service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:01:55.658051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-9b4bf32d-2037-4900-8c4d-96be7e4153da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:01:55.670583 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4bf32d-2037-4900-8c4d-96be7e4153da service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:01:55.672313 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4bf32d-2037-4900-8c4d-96be7e4153da service placement] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:01:55.683326 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1372/2722] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:01:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:02:00.477729 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386225c4-9c49-454a-8d0c-ab24c8555e68 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:02:00.478265 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386225c4-9c49-454a-8d0c-ab24c8555e68 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:02:00.478390 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386225c4-9c49-454a-8d0c-ab24c8555e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:02:00.510253 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-386225c4-9c49-454a-8d0c-ab24c8555e68 None None] MFA Rules not processed for user `83076e7b11fb446ca50f23fe8e6e7ab6`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:02:00.510576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-386225c4-9c49-454a-8d0c-ab24c8555e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:02:00.587038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1351/2723] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:02:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:02:00.589621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bf32d-2037-4900-8c4d-96be7e4153da service placement] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:02:00.603555 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bf32d-2037-4900-8c4d-96be7e4153da service placement] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:02:00.613207 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8978c1-f630-46b9-8f23-9f08b31432e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:02:00.614472 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8978c1-f630-46b9-8f23-9f08b31432e6 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:02:00.614910 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8978c1-f630-46b9-8f23-9f08b31432e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:02:00.615384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8978c1-f630-46b9-8f23-9f08b31432e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:02:00.617256 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8978c1-f630-46b9-8f23-9f08b31432e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:02:00.619522 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8978c1-f630-46b9-8f23-9f08b31432e6 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:02:00.740335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1373/2724] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:02:00 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:02:00.741863 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386225c4-9c49-454a-8d0c-ab24c8555e68 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:02:00.756865 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386225c4-9c49-454a-8d0c-ab24c8555e68 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:02:00.826697 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=100 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:02:00.833982 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b9e9c-b435-4409-abf5-c94e702a3a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:02:00.834805 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b9e9c-b435-4409-abf5-c94e702a3a78 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:02:00.835164 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b9e9c-b435-4409-abf5-c94e702a3a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:02:00.835507 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b9e9c-b435-4409-abf5-c94e702a3a78 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:02:00.836234 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b9e9c-b435-4409-abf5-c94e702a3a78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:02:00.838743 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b9e9c-b435-4409-abf5-c94e702a3a78 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:02:00.904394 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1352/2725] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:02:00 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 01:02:00.909180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8978c1-f630-46b9-8f23-9f08b31432e6 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:02:00.920365 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8978c1-f630-46b9-8f23-9f08b31432e6 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:02:00.927091 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb42c6-b0b0-4a89-89ae-c55d9ee41677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:02:00.927779 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb42c6-b0b0-4a89-89ae-c55d9ee41677 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:02:00.928198 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb42c6-b0b0-4a89-89ae-c55d9ee41677 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:02:00.928528 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb42c6-b0b0-4a89-89ae-c55d9ee41677 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:02:00.929185 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb42c6-b0b0-4a89-89ae-c55d9ee41677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:02:00.930786 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb42c6-b0b0-4a89-89ae-c55d9ee41677 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:02:00.943398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1374/2726] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:02:00.947719 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b9e9c-b435-4409-abf5-c94e702a3a78 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:02:00.958369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b9e9c-b435-4409-abf5-c94e702a3a78 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:02:00.963312 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e738d-0651-4d81-8b3c-65b7b4082fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:02:00.964102 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e738d-0651-4d81-8b3c-65b7b4082fe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:02:00.964517 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e738d-0651-4d81-8b3c-65b7b4082fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:02:00.964900 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e738d-0651-4d81-8b3c-65b7b4082fe7 admin admin] PATH_INFO: `/v3/projects/5d843f2ba1a24c9ca97a215c8cbe2240/users/d770a26f90c740a2bbfc2d4dee853da2/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:02:00.981837 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e738d-0651-4d81-8b3c-65b7b4082fe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d843f2ba1a24c9ca97a215c8cbe2240, user_id=d770a26f90c740a2bbfc2d4dee853da2, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:02:00.985471 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e738d-0651-4d81-8b3c-65b7b4082fe7 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:02:01.075408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1353/2727] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:02:00 2022] PUT /identity/v3/projects/5d843f2ba1a24c9ca97a215c8cbe2240/users/d770a26f90c740a2bbfc2d4dee853da2/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:02:01.079410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb42c6-b0b0-4a89-89ae-c55d9ee41677 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:02:01.089940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb42c6-b0b0-4a89-89ae-c55d9ee41677 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:02:01.158414 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c04e0-8999-4e40-9f34-02418178a2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:02:01.159134 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c04e0-8999-4e40-9f34-02418178a2b4 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:02:01.159488 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c04e0-8999-4e40-9f34-02418178a2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:02:01.159834 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c04e0-8999-4e40-9f34-02418178a2b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:02:01.160581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c04e0-8999-4e40-9f34-02418178a2b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:02:01.162512 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c04e0-8999-4e40-9f34-02418178a2b4 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:02:01.174238 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1375/2728] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:02:01.178482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e738d-0651-4d81-8b3c-65b7b4082fe7 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:02:01.189999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e738d-0651-4d81-8b3c-65b7b4082fe7 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:02:01.194970 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed05bc-9957-4057-a891-2a83d7245a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:02:01.195689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed05bc-9957-4057-a891-2a83d7245a54 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:02:01.195973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed05bc-9957-4057-a891-2a83d7245a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:02:01.196253 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed05bc-9957-4057-a891-2a83d7245a54 admin admin] PATH_INFO: `/v3/projects/5d843f2ba1a24c9ca97a215c8cbe2240/users/d770a26f90c740a2bbfc2d4dee853da2/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:02:01.198424 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed05bc-9957-4057-a891-2a83d7245a54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d843f2ba1a24c9ca97a215c8cbe2240, user_id=d770a26f90c740a2bbfc2d4dee853da2, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:02:01.201128 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed05bc-9957-4057-a891-2a83d7245a54 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:02:01.212589 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1354/2729] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:02:01 2022] PUT /identity/v3/projects/5d843f2ba1a24c9ca97a215c8cbe2240/users/d770a26f90c740a2bbfc2d4dee853da2/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:02:07.349240 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622a57b-cbf4-4762-8c2a-4ba42b458b30 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:02:07.350051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622a57b-cbf4-4762-8c2a-4ba42b458b30 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:02:07.350337 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622a57b-cbf4-4762-8c2a-4ba42b458b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:02:07.379644 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:02:07.384384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-0622a57b-cbf4-4762-8c2a-4ba42b458b30 None None] MFA Rules not processed for user `d770a26f90c740a2bbfc2d4dee853da2`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:02:07.385749 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-0622a57b-cbf4-4762-8c2a-4ba42b458b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:02:07.473420 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1376/2730] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:02:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:02:07.488111 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed05bc-9957-4057-a891-2a83d7245a54 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:02:07.490572 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-88ed05bc-9957-4057-a891-2a83d7245a54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:02:07.613091 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed05bc-9957-4057-a891-2a83d7245a54 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:02:07.619437 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe00f137-3d8a-4c82-a42a-ad85e3a34483 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:02:07.620063 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00f137-3d8a-4c82-a42a-ad85e3a34483 service nova] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:02:07.620350 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00f137-3d8a-4c82-a42a-ad85e3a34483 service nova] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:02:07.620668 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00f137-3d8a-4c82-a42a-ad85e3a34483 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:02:07.685944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe00f137-3d8a-4c82-a42a-ad85e3a34483 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:02:07.688131 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe00f137-3d8a-4c82-a42a-ad85e3a34483 service nova] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:02:07.699422 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1355/2731] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:02:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:02:09.620687 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3964d-cd75-41ed-b603-013455044a16 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:02:09.621375 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3964d-cd75-41ed-b603-013455044a16 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:02:09.622790 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3964d-cd75-41ed-b603-013455044a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:02:09.646265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-b0d3964d-cd75-41ed-b603-013455044a16 None None] MFA Rules not processed for user `38047508b7d44f05af7116a15ba7e319`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:02:09.654374 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-b0d3964d-cd75-41ed-b603-013455044a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:02:09.709072 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1377/2732] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:02:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:02:11.058618 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8e41a-3a4d-42ca-a57d-e1c105898762 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:02:11.059195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8e41a-3a4d-42ca-a57d-e1c105898762 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:02:11.059195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8e41a-3a4d-42ca-a57d-e1c105898762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:02:11.079841 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-edc8e41a-3a4d-42ca-a57d-e1c105898762 None None] MFA Rules not processed for user `83076e7b11fb446ca50f23fe8e6e7ab6`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:02:11.081118 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-edc8e41a-3a4d-42ca-a57d-e1c105898762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:02:11.088890 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1356/2733] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:02:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:02:19.701570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=53 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:02:20.614018 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3964d-cd75-41ed-b603-013455044a16 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:02:20.683388 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3964d-cd75-41ed-b603-013455044a16 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:02:20.688366 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1574c7ac-50ed-49e2-ad56-9c6527ef1def service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:02:20.689462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574c7ac-50ed-49e2-ad56-9c6527ef1def service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:02:20.700921 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574c7ac-50ed-49e2-ad56-9c6527ef1def service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:02:20.700921 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574c7ac-50ed-49e2-ad56-9c6527ef1def service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:02:20.700921 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-1574c7ac-50ed-49e2-ad56-9c6527ef1def service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:02:20.758976 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1574c7ac-50ed-49e2-ad56-9c6527ef1def service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:02:20.760441 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1574c7ac-50ed-49e2-ad56-9c6527ef1def service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:02:20.771382 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1378/2734] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:02:20 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:02:21.073616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:INSERT=5 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:02:30.767387 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=54 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:02:46.972863 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8e41a-3a4d-42ca-a57d-e1c105898762 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:02:46.983137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8e41a-3a4d-42ca-a57d-e1c105898762 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:02:46.986763 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9783ea-8325-40f0-b38a-6a4a2f1d8275 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:02:46.987365 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9783ea-8325-40f0-b38a-6a4a2f1d8275 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:02:46.987643 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9783ea-8325-40f0-b38a-6a4a2f1d8275 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:02:46.987914 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9783ea-8325-40f0-b38a-6a4a2f1d8275 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:02:46.996859 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9783ea-8325-40f0-b38a-6a4a2f1d8275 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:02:46.998329 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9783ea-8325-40f0-b38a-6a4a2f1d8275 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:02:47.007698 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1357/2735] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:02:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:02:57.003433 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=6 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:03:19.245469 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d51f2-e4dc-4d02-b460-3b5ec579f074 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:03:19.246107 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d51f2-e4dc-4d02-b460-3b5ec579f074 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:03:19.246107 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d51f2-e4dc-4d02-b460-3b5ec579f074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:03:19.266786 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-960d51f2-e4dc-4d02-b460-3b5ec579f074 None None] MFA Rules not processed for user `38047508b7d44f05af7116a15ba7e319`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:03:19.268184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-960d51f2-e4dc-4d02-b460-3b5ec579f074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:03:19.274985 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1379/2736] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:03:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:03:29.260928 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:04:03.550925 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9783ea-8325-40f0-b38a-6a4a2f1d8275 service neutron] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:04:03.552441 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-af9783ea-8325-40f0-b38a-6a4a2f1d8275 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:04:03.577080 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9783ea-8325-40f0-b38a-6a4a2f1d8275 service neutron] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:04:03.581330 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe236e6-1e37-477a-af17-e89baaf13717 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:04:03.581939 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe236e6-1e37-477a-af17-e89baaf13717 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:04:03.582220 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe236e6-1e37-477a-af17-e89baaf13717 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:04:03.582491 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe236e6-1e37-477a-af17-e89baaf13717 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:04:03.584224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-cfe236e6-1e37-477a-af17-e89baaf13717 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:04:03.598670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe236e6-1e37-477a-af17-e89baaf13717 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:04:03.600107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe236e6-1e37-477a-af17-e89baaf13717 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:04:03.611696 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1358/2737] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:04:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:04:04.797566 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d51f2-e4dc-4d02-b460-3b5ec579f074 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:04:04.807255 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d51f2-e4dc-4d02-b460-3b5ec579f074 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:04:04.811139 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985579ea-ab52-4d73-a759-9553e94e04f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:04:04.811754 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985579ea-ab52-4d73-a759-9553e94e04f3 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:04:04.813097 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985579ea-ab52-4d73-a759-9553e94e04f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:04:04.813097 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985579ea-ab52-4d73-a759-9553e94e04f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:04:04.814606 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-985579ea-ab52-4d73-a759-9553e94e04f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:04:04.828927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985579ea-ab52-4d73-a759-9553e94e04f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:04:04.830412 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985579ea-ab52-4d73-a759-9553e94e04f3 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:04:04.842004 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1380/2738] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:04:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:04:11.760825 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe236e6-1e37-477a-af17-e89baaf13717 service neutron] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:04:11.770814 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe236e6-1e37-477a-af17-e89baaf13717 service neutron] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:04:11.774932 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d2fe78-5a9a-46b3-93db-4d3f3bcf4ae3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:04:11.775619 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d2fe78-5a9a-46b3-93db-4d3f3bcf4ae3 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:04:11.775993 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d2fe78-5a9a-46b3-93db-4d3f3bcf4ae3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:04:11.776330 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d2fe78-5a9a-46b3-93db-4d3f3bcf4ae3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:04:11.778339 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-a1d2fe78-5a9a-46b3-93db-4d3f3bcf4ae3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:04:11.790252 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d2fe78-5a9a-46b3-93db-4d3f3bcf4ae3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:04:11.794116 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d2fe78-5a9a-46b3-93db-4d3f3bcf4ae3 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:04:11.804917 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1359/2739] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:04:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:04:14.837532 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=6 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:04:19.044622 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985579ea-ab52-4d73-a759-9553e94e04f3 service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:04:19.055636 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985579ea-ab52-4d73-a759-9553e94e04f3 service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:04:19.059696 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f9fff-4e2d-4f64-bd0b-2ab14e99125b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:04:19.060309 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f9fff-4e2d-4f64-bd0b-2ab14e99125b service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:04:19.060818 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f9fff-4e2d-4f64-bd0b-2ab14e99125b service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:04:19.061093 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f9fff-4e2d-4f64-bd0b-2ab14e99125b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:04:19.063855 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-6e0f9fff-4e2d-4f64-bd0b-2ab14e99125b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:04:19.128624 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0f9fff-4e2d-4f64-bd0b-2ab14e99125b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:04:19.130112 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0f9fff-4e2d-4f64-bd0b-2ab14e99125b service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:04:19.140059 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1381/2740] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:04:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:04:21.799717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=17 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:04:26.441864 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d2fe78-5a9a-46b3-93db-4d3f3bcf4ae3 service neutron] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:04:26.457132 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d2fe78-5a9a-46b3-93db-4d3f3bcf4ae3 service neutron] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:04:26.464374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b2e1d-26dd-4106-94fa-79d468d8bd75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:04:26.465402 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b2e1d-26dd-4106-94fa-79d468d8bd75 service nova] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:04:26.469275 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b2e1d-26dd-4106-94fa-79d468d8bd75 service nova] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:04:26.469774 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b2e1d-26dd-4106-94fa-79d468d8bd75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:04:26.471876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-cd7b2e1d-26dd-4106-94fa-79d468d8bd75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:04:26.486449 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b2e1d-26dd-4106-94fa-79d468d8bd75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:04:26.491964 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b2e1d-26dd-4106-94fa-79d468d8bd75 service nova] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:04:26.507731 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1360/2741] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:04:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:04:27.051687 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f9fff-4e2d-4f64-bd0b-2ab14e99125b service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:04:27.051687 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-6e0f9fff-4e2d-4f64-bd0b-2ab14e99125b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:04:27.066318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f9fff-4e2d-4f64-bd0b-2ab14e99125b service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:04:27.071428 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b0477-1622-43e9-8615-a9f750447aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:04:27.072077 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b0477-1622-43e9-8615-a9f750447aa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:04:27.072360 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b0477-1622-43e9-8615-a9f750447aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:04:27.072676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b0477-1622-43e9-8615-a9f750447aa4 admin admin] PATH_INFO: `/v3/users/cd45f1813efc4926a30454aaa108ac47` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:04:27.073948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b0477-1622-43e9-8615-a9f750447aa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd45f1813efc4926a30454aaa108ac47)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:04:27.075951 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b0477-1622-43e9-8615-a9f750447aa4 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:04:27.320066 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-069b0477-1622-43e9-8615-a9f750447aa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd45f1813efc4926a30454aaa108ac47'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:04:27.320897 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-069b0477-1622-43e9-8615-a9f750447aa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd45f1813efc4926a30454aaa108ac47'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:04:27.332817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-069b0477-1622-43e9-8615-a9f750447aa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd45f1813efc4926a30454aaa108ac47'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:04:27.425834 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-069b0477-1622-43e9-8615-a9f750447aa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd45f1813efc4926a30454aaa108ac47'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:04:27.426457 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-069b0477-1622-43e9-8615-a9f750447aa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd45f1813efc4926a30454aaa108ac47'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:04:27.443172 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1382/2742] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:04:27 2022] DELETE /identity/v3/users/cd45f1813efc4926a30454aaa108ac47 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:04:27.701328 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b2e1d-26dd-4106-94fa-79d468d8bd75 service nova] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:04:27.712291 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b2e1d-26dd-4106-94fa-79d468d8bd75 service nova] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:04:27.774411 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd4172e-a5df-4828-9f57-4f1cd57ce27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:04:27.775057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4172e-a5df-4828-9f57-4f1cd57ce27c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:04:27.775335 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4172e-a5df-4828-9f57-4f1cd57ce27c admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:04:27.775609 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4172e-a5df-4828-9f57-4f1cd57ce27c admin admin] PATH_INFO: `/v3/projects/ca3e24d51bdf44c5ad7219fa57d3d23b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:04:27.776792 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd4172e-a5df-4828-9f57-4f1cd57ce27c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca3e24d51bdf44c5ad7219fa57d3d23b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:04:27.778775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd4172e-a5df-4828-9f57-4f1cd57ce27c admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:04:27.876817 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-8dd4172e-a5df-4828-9f57-4f1cd57ce27c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca3e24d51bdf44c5ad7219fa57d3d23b', 'request_id': 'req-8dd4172e-a5df-4828-9f57-4f1cd57ce27c'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:04:27.950113 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-8dd4172e-a5df-4828-9f57-4f1cd57ce27c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca3e24d51bdf44c5ad7219fa57d3d23b', 'request_id': 'req-8dd4172e-a5df-4828-9f57-4f1cd57ce27c'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:04:27.959707 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-8dd4172e-a5df-4828-9f57-4f1cd57ce27c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca3e24d51bdf44c5ad7219fa57d3d23b', 'request_id': 'req-8dd4172e-a5df-4828-9f57-4f1cd57ce27c'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:04:27.974727 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-8dd4172e-a5df-4828-9f57-4f1cd57ce27c admin admin] The token cache is being invalidate because project ca3e24d51bdf44c5ad7219fa57d3d23b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:04:27.975018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-8dd4172e-a5df-4828-9f57-4f1cd57ce27c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:04:27.975684 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-8dd4172e-a5df-4828-9f57-4f1cd57ce27c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:04:27.977248 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1361/2743] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:04:27 2022] DELETE /identity/v3/projects/ca3e24d51bdf44c5ad7219fa57d3d23b => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:04:27.981231 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b0477-1622-43e9-8615-a9f750447aa4 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:04:27.982669 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-069b0477-1622-43e9-8615-a9f750447aa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:04:27.998132 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b0477-1622-43e9-8615-a9f750447aa4 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:04:28.055224 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5391868-6f88-4de3-8303-f1f6c7e263d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:04:28.055902 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5391868-6f88-4de3-8303-f1f6c7e263d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:04:28.056165 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5391868-6f88-4de3-8303-f1f6c7e263d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:04:28.056450 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5391868-6f88-4de3-8303-f1f6c7e263d0 admin admin] PATH_INFO: `/v3/users/b7e3d19238f64065b3191400f12534b6` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:04:28.057686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5391868-6f88-4de3-8303-f1f6c7e263d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7e3d19238f64065b3191400f12534b6)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:04:28.059693 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5391868-6f88-4de3-8303-f1f6c7e263d0 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:04:28.066236 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:04:28.264870 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-b5391868-6f88-4de3-8303-f1f6c7e263d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7e3d19238f64065b3191400f12534b6'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:04:28.265787 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-b5391868-6f88-4de3-8303-f1f6c7e263d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7e3d19238f64065b3191400f12534b6'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:04:28.282966 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-b5391868-6f88-4de3-8303-f1f6c7e263d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7e3d19238f64065b3191400f12534b6'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:04:28.366862 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-b5391868-6f88-4de3-8303-f1f6c7e263d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7e3d19238f64065b3191400f12534b6'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:04:28.367491 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-b5391868-6f88-4de3-8303-f1f6c7e263d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7e3d19238f64065b3191400f12534b6'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:04:28.381176 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1383/2744] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:04:27 2022] DELETE /identity/v3/users/b7e3d19238f64065b3191400f12534b6 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:04:28.690274 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd4172e-a5df-4828-9f57-4f1cd57ce27c admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:04:28.691980 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-8dd4172e-a5df-4828-9f57-4f1cd57ce27c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:04:28.708387 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd4172e-a5df-4828-9f57-4f1cd57ce27c admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:04:28.771483 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b5452-1d6e-4446-b7c9-d2b3a2da10a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:04:28.772066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b5452-1d6e-4446-b7c9-d2b3a2da10a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:04:28.772348 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b5452-1d6e-4446-b7c9-d2b3a2da10a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:04:28.772654 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b5452-1d6e-4446-b7c9-d2b3a2da10a8 admin admin] PATH_INFO: `/v3/projects/016ffb0e1f50431299ecff65e07bd180` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:04:28.773793 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b5452-1d6e-4446-b7c9-d2b3a2da10a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=016ffb0e1f50431299ecff65e07bd180)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:04:28.775732 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b5452-1d6e-4446-b7c9-d2b3a2da10a8 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:04:28.800508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:04:28.899875 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1c9b5452-1d6e-4446-b7c9-d2b3a2da10a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '016ffb0e1f50431299ecff65e07bd180', 'request_id': 'req-1c9b5452-1d6e-4446-b7c9-d2b3a2da10a8'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:04:28.974093 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1c9b5452-1d6e-4446-b7c9-d2b3a2da10a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '016ffb0e1f50431299ecff65e07bd180', 'request_id': 'req-1c9b5452-1d6e-4446-b7c9-d2b3a2da10a8'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:04:28.981014 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1c9b5452-1d6e-4446-b7c9-d2b3a2da10a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '016ffb0e1f50431299ecff65e07bd180', 'request_id': 'req-1c9b5452-1d6e-4446-b7c9-d2b3a2da10a8'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:04:28.992040 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1c9b5452-1d6e-4446-b7c9-d2b3a2da10a8 admin admin] The token cache is being invalidate because project 016ffb0e1f50431299ecff65e07bd180 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:04:28.992602 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1c9b5452-1d6e-4446-b7c9-d2b3a2da10a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:04:28.993319 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-1c9b5452-1d6e-4446-b7c9-d2b3a2da10a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:04:28.995042 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1362/2745] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:04:28 2022] DELETE /identity/v3/projects/016ffb0e1f50431299ecff65e07bd180 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:04:31.381694 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d7901-4e13-4487-93cc-3c09b7afaa65 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:04:31.382231 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d7901-4e13-4487-93cc-3c09b7afaa65 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:04:31.382231 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d7901-4e13-4487-93cc-3c09b7afaa65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:04:31.401177 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-358d7901-4e13-4487-93cc-3c09b7afaa65 None None] MFA Rules not processed for user `38047508b7d44f05af7116a15ba7e319`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:04:31.402401 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-358d7901-4e13-4487-93cc-3c09b7afaa65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:04:31.459960 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1384/2746] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:04:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:04:31.765300 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b5452-1d6e-4446-b7c9-d2b3a2da10a8 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:04:31.767262 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-1c9b5452-1d6e-4446-b7c9-d2b3a2da10a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:04:31.822863 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b5452-1d6e-4446-b7c9-d2b3a2da10a8 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:04:31.826444 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274bb144-18db-449d-8436-3694beda65ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:04:31.827039 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274bb144-18db-449d-8436-3694beda65ad service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:04:31.827316 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274bb144-18db-449d-8436-3694beda65ad service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:04:31.827585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274bb144-18db-449d-8436-3694beda65ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:04:31.829815 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-274bb144-18db-449d-8436-3694beda65ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:04:31.839139 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274bb144-18db-449d-8436-3694beda65ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:04:31.840595 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274bb144-18db-449d-8436-3694beda65ad service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:04:31.848942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1363/2747] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:04:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:04:32.257987 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d7901-4e13-4487-93cc-3c09b7afaa65 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:04:32.259398 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-358d7901-4e13-4487-93cc-3c09b7afaa65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:04:32.333882 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d7901-4e13-4487-93cc-3c09b7afaa65 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:04:32.338896 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb665f7-5259-432d-a16a-2af42bb5a18d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2876f4182f0949039b862f053533fe85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:04:32.339508 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb665f7-5259-432d-a16a-2af42bb5a18d service placement] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:04:32.339819 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb665f7-5259-432d-a16a-2af42bb5a18d service placement] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:04:32.340093 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb665f7-5259-432d-a16a-2af42bb5a18d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:04:32.341881 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-2cb665f7-5259-432d-a16a-2af42bb5a18d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:04:32.353029 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb665f7-5259-432d-a16a-2af42bb5a18d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:04:32.354607 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb665f7-5259-432d-a16a-2af42bb5a18d service placement] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:04:32.364346 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1385/2748] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:04:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:04:41.844830 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:04:41.875254 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274bb144-18db-449d-8436-3694beda65ad service neutron] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:04:41.876737 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-274bb144-18db-449d-8436-3694beda65ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:04:41.948962 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274bb144-18db-449d-8436-3694beda65ad service neutron] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:04:41.954343 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31ae95-30d4-4a16-9e8e-76492f691ae1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:04:41.954948 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31ae95-30d4-4a16-9e8e-76492f691ae1 service nova] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:04:41.955233 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31ae95-30d4-4a16-9e8e-76492f691ae1 service nova] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:04:41.955508 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31ae95-30d4-4a16-9e8e-76492f691ae1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:04:41.957343 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-dc31ae95-30d4-4a16-9e8e-76492f691ae1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:04:41.969727 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31ae95-30d4-4a16-9e8e-76492f691ae1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:04:41.973752 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31ae95-30d4-4a16-9e8e-76492f691ae1 service nova] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:04:41.983625 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1364/2749] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:04:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:04:42.360167 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=80,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:04:52.030956 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=34 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:05:01.832225 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb665f7-5259-432d-a16a-2af42bb5a18d service placement] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:01.842055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb665f7-5259-432d-a16a-2af42bb5a18d service placement] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:01.845825 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dea1bf-bbf9-4fae-a943-f043100a2aee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:01.846334 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dea1bf-bbf9-4fae-a943-f043100a2aee service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:01.846673 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dea1bf-bbf9-4fae-a943-f043100a2aee service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:01.846885 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dea1bf-bbf9-4fae-a943-f043100a2aee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:01.848691 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-50dea1bf-bbf9-4fae-a943-f043100a2aee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:05:01.876731 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dea1bf-bbf9-4fae-a943-f043100a2aee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:01.876731 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dea1bf-bbf9-4fae-a943-f043100a2aee service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:01.884050 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1386/2750] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:05:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:05:01.960543 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31ae95-30d4-4a16-9e8e-76492f691ae1 service nova] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:01.960543 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-dc31ae95-30d4-4a16-9e8e-76492f691ae1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:05:01.971212 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31ae95-30d4-4a16-9e8e-76492f691ae1 service nova] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:01.975352 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18eef1c-1e5f-4242-aa03-b25b6e1d118e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:01.975954 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18eef1c-1e5f-4242-aa03-b25b6e1d118e service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:01.976231 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18eef1c-1e5f-4242-aa03-b25b6e1d118e service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:01.976531 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18eef1c-1e5f-4242-aa03-b25b6e1d118e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:01.978222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-a18eef1c-1e5f-4242-aa03-b25b6e1d118e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:05:01.992874 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18eef1c-1e5f-4242-aa03-b25b6e1d118e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:01.994311 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18eef1c-1e5f-4242-aa03-b25b6e1d118e service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:02.005671 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1365/2751] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:05:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:05:11.879744 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=6 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:05:12.000659 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=6 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:05:49.058621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dea1bf-bbf9-4fae-a943-f043100a2aee service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:49.060763 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-50dea1bf-bbf9-4fae-a943-f043100a2aee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:05:49.076499 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dea1bf-bbf9-4fae-a943-f043100a2aee service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:49.081661 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeddec33-f56c-4774-9ce2-3861163df9b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2876f4182f0949039b862f053533fe85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:49.082275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeddec33-f56c-4774-9ce2-3861163df9b2 service placement] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:49.082561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeddec33-f56c-4774-9ce2-3861163df9b2 service placement] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:49.082839 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeddec33-f56c-4774-9ce2-3861163df9b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:49.084617 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-aeddec33-f56c-4774-9ce2-3861163df9b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:05:49.097013 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeddec33-f56c-4774-9ce2-3861163df9b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:49.098618 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeddec33-f56c-4774-9ce2-3861163df9b2 service placement] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:49.110432 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1387/2752] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:05:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:05:49.225194 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18eef1c-1e5f-4242-aa03-b25b6e1d118e service neutron] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:49.226810 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-a18eef1c-1e5f-4242-aa03-b25b6e1d118e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:05:49.293543 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18eef1c-1e5f-4242-aa03-b25b6e1d118e service neutron] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:49.298063 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff9f81-0b73-4bf9-9ca5-bb947f6287b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:49.298691 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff9f81-0b73-4bf9-9ca5-bb947f6287b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:49.298974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff9f81-0b73-4bf9-9ca5-bb947f6287b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:49.299249 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff9f81-0b73-4bf9-9ca5-bb947f6287b4 admin admin] PATH_INFO: `/v3/users/38047508b7d44f05af7116a15ba7e319` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:49.300428 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff9f81-0b73-4bf9-9ca5-bb947f6287b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38047508b7d44f05af7116a15ba7e319)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:49.302370 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff9f81-0b73-4bf9-9ca5-bb947f6287b4 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:49.614328 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-3cff9f81-0b73-4bf9-9ca5-bb947f6287b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38047508b7d44f05af7116a15ba7e319'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:49.625721 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-3cff9f81-0b73-4bf9-9ca5-bb947f6287b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38047508b7d44f05af7116a15ba7e319'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:49.736910 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-3cff9f81-0b73-4bf9-9ca5-bb947f6287b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38047508b7d44f05af7116a15ba7e319'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:49.737529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-3cff9f81-0b73-4bf9-9ca5-bb947f6287b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38047508b7d44f05af7116a15ba7e319'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:49.738156 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-3cff9f81-0b73-4bf9-9ca5-bb947f6287b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38047508b7d44f05af7116a15ba7e319'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:49.750872 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1366/2753] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:05:49 2022] DELETE /identity/v3/users/38047508b7d44f05af7116a15ba7e319 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:05:50.030942 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeddec33-f56c-4774-9ce2-3861163df9b2 service placement] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:50.051893 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeddec33-f56c-4774-9ce2-3861163df9b2 service placement] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:50.110621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743bcdf-5621-40d6-a988-ccd8abb2b4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:50.111318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743bcdf-5621-40d6-a988-ccd8abb2b4c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:50.111525 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743bcdf-5621-40d6-a988-ccd8abb2b4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:50.111800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743bcdf-5621-40d6-a988-ccd8abb2b4c6 admin admin] PATH_INFO: `/v3/projects/03b021c415a147e08dbfede2ddc8cc27` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:50.112955 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743bcdf-5621-40d6-a988-ccd8abb2b4c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03b021c415a147e08dbfede2ddc8cc27)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:50.114866 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743bcdf-5621-40d6-a988-ccd8abb2b4c6 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:50.277460 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-7743bcdf-5621-40d6-a988-ccd8abb2b4c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03b021c415a147e08dbfede2ddc8cc27', 'request_id': 'req-7743bcdf-5621-40d6-a988-ccd8abb2b4c6'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:50.360939 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-7743bcdf-5621-40d6-a988-ccd8abb2b4c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03b021c415a147e08dbfede2ddc8cc27', 'request_id': 'req-7743bcdf-5621-40d6-a988-ccd8abb2b4c6'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:50.367236 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-7743bcdf-5621-40d6-a988-ccd8abb2b4c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03b021c415a147e08dbfede2ddc8cc27', 'request_id': 'req-7743bcdf-5621-40d6-a988-ccd8abb2b4c6'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:50.378580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-7743bcdf-5621-40d6-a988-ccd8abb2b4c6 admin admin] The token cache is being invalidate because project 03b021c415a147e08dbfede2ddc8cc27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:05:50.378878 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-7743bcdf-5621-40d6-a988-ccd8abb2b4c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:50.379513 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-7743bcdf-5621-40d6-a988-ccd8abb2b4c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:50.381301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1388/2754] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:05:50 2022] DELETE /identity/v3/projects/03b021c415a147e08dbfede2ddc8cc27 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:05:50.386026 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95911380-c2bc-408e-ba73-c087dbb559c9 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:50.386512 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95911380-c2bc-408e-ba73-c087dbb559c9 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:50.386621 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95911380-c2bc-408e-ba73-c087dbb559c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:50.407899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-95911380-c2bc-408e-ba73-c087dbb559c9 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:05:50.409142 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-95911380-c2bc-408e-ba73-c087dbb559c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:05:50.472442 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1367/2755] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:05:50.481275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd1942-c8a4-4454-9c45-783f82493eb4 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:50.481552 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd1942-c8a4-4454-9c45-783f82493eb4 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:50.481819 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd1942-c8a4-4454-9c45-783f82493eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:50.502729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-10cd1942-c8a4-4454-9c45-783f82493eb4 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:05:50.503934 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-10cd1942-c8a4-4454-9c45-783f82493eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:05:50.511778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1389/2756] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:05:50.516569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95911380-c2bc-408e-ba73-c087dbb559c9 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:50.526343 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95911380-c2bc-408e-ba73-c087dbb559c9 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:50.531559 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49ac25-8194-4e74-b5bf-370521399f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:50.532109 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49ac25-8194-4e74-b5bf-370521399f99 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:50.532377 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49ac25-8194-4e74-b5bf-370521399f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:50.532680 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49ac25-8194-4e74-b5bf-370521399f99 admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:50.533325 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce49ac25-8194-4e74-b5bf-370521399f99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 01:05:50.533626 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce49ac25-8194-4e74-b5bf-370521399f99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:50.535224 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce49ac25-8194-4e74-b5bf-370521399f99 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:50.547028 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1368/2757] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 01:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 01:05:50.551280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cd1942-c8a4-4454-9c45-783f82493eb4 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:50.560463 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cd1942-c8a4-4454-9c45-783f82493eb4 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:50.565912 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfdcd2-19c6-4420-be6b-d7f3efc02921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:50.566547 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfdcd2-19c6-4420-be6b-d7f3efc02921 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:50.566854 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfdcd2-19c6-4420-be6b-d7f3efc02921 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:50.567151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfdcd2-19c6-4420-be6b-d7f3efc02921 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:50.567821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bfdcd2-19c6-4420-be6b-d7f3efc02921 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:50.569766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bfdcd2-19c6-4420-be6b-d7f3efc02921 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:50.718332 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1390/2758] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:05:50 2022] POST /identity/v3/projects => generated 367 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:05:50.722724 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49ac25-8194-4e74-b5bf-370521399f99 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:50.732738 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49ac25-8194-4e74-b5bf-370521399f99 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:50.738916 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:05:50.792242 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449e8b5-981c-4fc7-840b-8f43c966b0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:50.792819 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449e8b5-981c-4fc7-840b-8f43c966b0d6 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:50.793708 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449e8b5-981c-4fc7-840b-8f43c966b0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:50.793983 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449e8b5-981c-4fc7-840b-8f43c966b0d6 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:50.794624 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449e8b5-981c-4fc7-840b-8f43c966b0d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:50.796631 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449e8b5-981c-4fc7-840b-8f43c966b0d6 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:50.923351 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1369/2759] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:05:50 2022] POST /identity/v3/users => generated 284 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:05:50.927198 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfdcd2-19c6-4420-be6b-d7f3efc02921 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:50.938138 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfdcd2-19c6-4420-be6b-d7f3efc02921 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:50.942982 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69d7d1-ed60-4e2a-832e-880619005150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:50.953343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69d7d1-ed60-4e2a-832e-880619005150 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:50.953343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69d7d1-ed60-4e2a-832e-880619005150 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:50.953343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69d7d1-ed60-4e2a-832e-880619005150 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:50.953343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69d7d1-ed60-4e2a-832e-880619005150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:50.953343 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69d7d1-ed60-4e2a-832e-880619005150 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:50.957806 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1391/2760] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:05:50.961780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449e8b5-981c-4fc7-840b-8f43c966b0d6 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:50.972730 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449e8b5-981c-4fc7-840b-8f43c966b0d6 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:50.977412 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa05ea-9bcc-42bb-a416-26afe7a44dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:50.978023 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa05ea-9bcc-42bb-a416-26afe7a44dca admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:50.978308 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa05ea-9bcc-42bb-a416-26afe7a44dca admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:50.978585 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa05ea-9bcc-42bb-a416-26afe7a44dca admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:50.979264 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa05ea-9bcc-42bb-a416-26afe7a44dca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:50.980944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa05ea-9bcc-42bb-a416-26afe7a44dca admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:50.991834 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1370/2761] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:05:50.996134 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69d7d1-ed60-4e2a-832e-880619005150 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:51.006363 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69d7d1-ed60-4e2a-832e-880619005150 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:51.010948 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a89073-d564-4148-8e3d-fcdc8c23bb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:51.011604 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a89073-d564-4148-8e3d-fcdc8c23bb45 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:51.011888 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a89073-d564-4148-8e3d-fcdc8c23bb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:51.012166 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a89073-d564-4148-8e3d-fcdc8c23bb45 admin admin] PATH_INFO: `/v3/projects/96fca4fdac8a432f9e5dff1fbc9c5b1b/users/36ac135954874beb8f880dfd1368c6a1/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:51.027595 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a89073-d564-4148-8e3d-fcdc8c23bb45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96fca4fdac8a432f9e5dff1fbc9c5b1b, user_id=36ac135954874beb8f880dfd1368c6a1, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:51.030860 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a89073-d564-4148-8e3d-fcdc8c23bb45 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:51.132376 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1392/2762] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:05:50 2022] PUT /identity/v3/projects/96fca4fdac8a432f9e5dff1fbc9c5b1b/users/36ac135954874beb8f880dfd1368c6a1/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 01:05:51.144743 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a5459-adae-4570-bccc-3acd2aca39c1 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:51.145106 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a5459-adae-4570-bccc-3acd2aca39c1 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:51.145444 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a5459-adae-4570-bccc-3acd2aca39c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:51.173823 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-964a5459-adae-4570-bccc-3acd2aca39c1 None None] MFA Rules not processed for user `36ac135954874beb8f880dfd1368c6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:05:51.175031 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-964a5459-adae-4570-bccc-3acd2aca39c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:05:51.237460 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1371/2763] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:05:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:05:51.241940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a89073-d564-4148-8e3d-fcdc8c23bb45 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:51.247767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:05:51.305262 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a89073-d564-4148-8e3d-fcdc8c23bb45 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:51.309932 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3b629-3660-483f-b0eb-00dcf6ef52d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:51.310570 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3b629-3660-483f-b0eb-00dcf6ef52d9 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:51.310851 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3b629-3660-483f-b0eb-00dcf6ef52d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:51.311671 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3b629-3660-483f-b0eb-00dcf6ef52d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:51.312292 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3b629-3660-483f-b0eb-00dcf6ef52d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:51.314278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3b629-3660-483f-b0eb-00dcf6ef52d9 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:51.494690 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1393/2764] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:05:51 2022] POST /identity/v3/projects => generated 369 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:05:51.498781 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a5459-adae-4570-bccc-3acd2aca39c1 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:51.509842 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a5459-adae-4570-bccc-3acd2aca39c1 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:51.565023 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966f365-1ad2-4565-a375-e1138119fa3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:51.565636 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966f365-1ad2-4565-a375-e1138119fa3d admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:51.565921 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966f365-1ad2-4565-a375-e1138119fa3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:51.566200 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966f365-1ad2-4565-a375-e1138119fa3d admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:51.566825 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966f365-1ad2-4565-a375-e1138119fa3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:51.568775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966f365-1ad2-4565-a375-e1138119fa3d admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:51.675560 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1372/2765] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:05:51 2022] POST /identity/v3/users => generated 291 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:05:51.679571 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3b629-3660-483f-b0eb-00dcf6ef52d9 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:51.690834 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3b629-3660-483f-b0eb-00dcf6ef52d9 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:51.695557 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e2d2c-5c12-447e-845b-32a3c62060f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:51.696162 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e2d2c-5c12-447e-845b-32a3c62060f7 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:51.696441 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e2d2c-5c12-447e-845b-32a3c62060f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:51.696812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e2d2c-5c12-447e-845b-32a3c62060f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:51.697457 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e2d2c-5c12-447e-845b-32a3c62060f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:51.699072 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e2d2c-5c12-447e-845b-32a3c62060f7 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:51.709568 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1394/2766] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:05:51.713760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966f365-1ad2-4565-a375-e1138119fa3d admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:51.720130 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:05:51.723770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966f365-1ad2-4565-a375-e1138119fa3d admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:51.728426 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8a7ea-d7ed-4386-be61-342d24f0dce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:51.729116 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8a7ea-d7ed-4386-be61-342d24f0dce2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:51.729397 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8a7ea-d7ed-4386-be61-342d24f0dce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:51.729671 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8a7ea-d7ed-4386-be61-342d24f0dce2 admin admin] PATH_INFO: `/v3/projects/f28bd8721a16436883c20dc85553bdc2/users/5281843322b14bbab3715f6ac8531da5/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:51.744575 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8a7ea-d7ed-4386-be61-342d24f0dce2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f28bd8721a16436883c20dc85553bdc2, user_id=5281843322b14bbab3715f6ac8531da5, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:51.747212 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8a7ea-d7ed-4386-be61-342d24f0dce2 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:51.944642 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1373/2767] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:05:51 2022] PUT /identity/v3/projects/f28bd8721a16436883c20dc85553bdc2/users/5281843322b14bbab3715f6ac8531da5/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:05:51.976810 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e2d2c-5c12-447e-845b-32a3c62060f7 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:51.976810 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e2d2c-5c12-447e-845b-32a3c62060f7 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:52.014922 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0d6e5-df68-4c8f-95c8-fad818ae0421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:52.015549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0d6e5-df68-4c8f-95c8-fad818ae0421 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:52.015862 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0d6e5-df68-4c8f-95c8-fad818ae0421 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:52.016144 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0d6e5-df68-4c8f-95c8-fad818ae0421 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:52.016808 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b0d6e5-df68-4c8f-95c8-fad818ae0421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:52.018405 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b0d6e5-df68-4c8f-95c8-fad818ae0421 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:52.027816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1395/2768] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:05:52.032673 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8a7ea-d7ed-4386-be61-342d24f0dce2 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:52.043143 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8a7ea-d7ed-4386-be61-342d24f0dce2 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:52.047818 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fdde00-d7c7-40eb-b0f7-a53e076f5ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:52.051090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fdde00-d7c7-40eb-b0f7-a53e076f5ae1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:52.051090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fdde00-d7c7-40eb-b0f7-a53e076f5ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:52.051090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fdde00-d7c7-40eb-b0f7-a53e076f5ae1 admin admin] PATH_INFO: `/v3/projects/f28bd8721a16436883c20dc85553bdc2/users/5281843322b14bbab3715f6ac8531da5/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:52.051090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fdde00-d7c7-40eb-b0f7-a53e076f5ae1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f28bd8721a16436883c20dc85553bdc2, user_id=5281843322b14bbab3715f6ac8531da5, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:52.053573 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fdde00-d7c7-40eb-b0f7-a53e076f5ae1 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:52.063959 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1374/2769] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:05:52 2022] PUT /identity/v3/projects/f28bd8721a16436883c20dc85553bdc2/users/5281843322b14bbab3715f6ac8531da5/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:05:52.074361 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1239f4c-d6aa-40a2-8cdb-7569ed38ff81 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:52.074700 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1239f4c-d6aa-40a2-8cdb-7569ed38ff81 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:52.074983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1239f4c-d6aa-40a2-8cdb-7569ed38ff81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:52.102447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-a1239f4c-d6aa-40a2-8cdb-7569ed38ff81 None None] MFA Rules not processed for user `5281843322b14bbab3715f6ac8531da5`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:05:52.103638 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a1239f4c-d6aa-40a2-8cdb-7569ed38ff81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:05:52.169615 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1396/2770] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:05:52 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 01:05:52.180445 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fdde00-d7c7-40eb-b0f7-a53e076f5ae1 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:52.182169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-a4fdde00-d7c7-40eb-b0f7-a53e076f5ae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:05:52.211241 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:05:52.258482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fdde00-d7c7-40eb-b0f7-a53e076f5ae1 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:52.262104 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1965f-8c6b-4bb9-a6b9-8ddcc09be7f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:52.262702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1965f-8c6b-4bb9-a6b9-8ddcc09be7f3 service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:52.262985 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1965f-8c6b-4bb9-a6b9-8ddcc09be7f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:52.263258 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1965f-8c6b-4bb9-a6b9-8ddcc09be7f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:52.272607 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1965f-8c6b-4bb9-a6b9-8ddcc09be7f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:52.274038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1965f-8c6b-4bb9-a6b9-8ddcc09be7f3 service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:52.283775 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1375/2771] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:05:52 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:05:55.610800 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1239f4c-d6aa-40a2-8cdb-7569ed38ff81 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:55.678621 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1239f4c-d6aa-40a2-8cdb-7569ed38ff81 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:55.683144 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21db26c-573f-4e06-ad50-295f3402833d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:55.683770 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21db26c-573f-4e06-ad50-295f3402833d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:55.684047 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21db26c-573f-4e06-ad50-295f3402833d admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:55.684319 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21db26c-573f-4e06-ad50-295f3402833d admin admin] PATH_INFO: `/v3/users/36ac135954874beb8f880dfd1368c6a1` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:55.685465 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21db26c-573f-4e06-ad50-295f3402833d admin admin] RBAC: Authorizing `identity:delete_user(user_id=36ac135954874beb8f880dfd1368c6a1)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:55.692546 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21db26c-573f-4e06-ad50-295f3402833d admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:55.879858 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-a21db26c-573f-4e06-ad50-295f3402833d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36ac135954874beb8f880dfd1368c6a1'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:55.880525 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-a21db26c-573f-4e06-ad50-295f3402833d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36ac135954874beb8f880dfd1368c6a1'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:55.892626 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-a21db26c-573f-4e06-ad50-295f3402833d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36ac135954874beb8f880dfd1368c6a1'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:55.972560 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-a21db26c-573f-4e06-ad50-295f3402833d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36ac135954874beb8f880dfd1368c6a1'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:55.973205 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-a21db26c-573f-4e06-ad50-295f3402833d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36ac135954874beb8f880dfd1368c6a1'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:55.986284 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1397/2772] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:05:55 2022] DELETE /identity/v3/users/36ac135954874beb8f880dfd1368c6a1 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 01:05:55.990785 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad496dbe-60cb-4e37-87c2-b1a3e90ca845 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:55.991273 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad496dbe-60cb-4e37-87c2-b1a3e90ca845 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:55.991388 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad496dbe-60cb-4e37-87c2-b1a3e90ca845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:56.010705 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-ad496dbe-60cb-4e37-87c2-b1a3e90ca845 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:05:56.011918 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-ad496dbe-60cb-4e37-87c2-b1a3e90ca845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:05:56.070222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1376/2773] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:05:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:05:56.080069 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21db26c-573f-4e06-ad50-295f3402833d admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:56.081553 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a21db26c-573f-4e06-ad50-295f3402833d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:05:56.137785 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21db26c-573f-4e06-ad50-295f3402833d admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:56.141392 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89508430-fd19-4a41-88c1-9c0bb942e157 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:56.142000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89508430-fd19-4a41-88c1-9c0bb942e157 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:56.142280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89508430-fd19-4a41-88c1-9c0bb942e157 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:56.142549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89508430-fd19-4a41-88c1-9c0bb942e157 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:56.151698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89508430-fd19-4a41-88c1-9c0bb942e157 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:56.153182 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89508430-fd19-4a41-88c1-9c0bb942e157 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:56.163353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1398/2774] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:05:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:05:56.594734 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad496dbe-60cb-4e37-87c2-b1a3e90ca845 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:56.594994 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-ad496dbe-60cb-4e37-87c2-b1a3e90ca845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:05:56.610988 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad496dbe-60cb-4e37-87c2-b1a3e90ca845 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:56.622797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b1d31c-970e-48d8-bccc-43c88655c6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:56.623944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1d31c-970e-48d8-bccc-43c88655c6a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:56.624225 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1d31c-970e-48d8-bccc-43c88655c6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:56.624557 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1d31c-970e-48d8-bccc-43c88655c6a6 admin admin] PATH_INFO: `/v3/projects/96fca4fdac8a432f9e5dff1fbc9c5b1b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:56.625695 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b1d31c-970e-48d8-bccc-43c88655c6a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96fca4fdac8a432f9e5dff1fbc9c5b1b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:56.627594 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b1d31c-970e-48d8-bccc-43c88655c6a6 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:56.746997 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:05:56.750299 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-37b1d31c-970e-48d8-bccc-43c88655c6a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96fca4fdac8a432f9e5dff1fbc9c5b1b', 'request_id': 'req-37b1d31c-970e-48d8-bccc-43c88655c6a6'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:56.834817 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-37b1d31c-970e-48d8-bccc-43c88655c6a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96fca4fdac8a432f9e5dff1fbc9c5b1b', 'request_id': 'req-37b1d31c-970e-48d8-bccc-43c88655c6a6'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:56.841320 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-37b1d31c-970e-48d8-bccc-43c88655c6a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96fca4fdac8a432f9e5dff1fbc9c5b1b', 'request_id': 'req-37b1d31c-970e-48d8-bccc-43c88655c6a6'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:56.851615 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-37b1d31c-970e-48d8-bccc-43c88655c6a6 admin admin] The token cache is being invalidate because project 96fca4fdac8a432f9e5dff1fbc9c5b1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:05:56.851901 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-37b1d31c-970e-48d8-bccc-43c88655c6a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:56.852568 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-37b1d31c-970e-48d8-bccc-43c88655c6a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:56.854074 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1377/2775] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:05:56 2022] DELETE /identity/v3/projects/96fca4fdac8a432f9e5dff1fbc9c5b1b => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:05:56.858227 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89508430-fd19-4a41-88c1-9c0bb942e157 service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:56.868280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89508430-fd19-4a41-88c1-9c0bb942e157 service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:56.887204 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:05:56.926408 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e088a37-d526-46a4-9f56-220e28f71c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:56.927041 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e088a37-d526-46a4-9f56-220e28f71c13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:56.927320 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e088a37-d526-46a4-9f56-220e28f71c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:56.927596 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e088a37-d526-46a4-9f56-220e28f71c13 admin admin] PATH_INFO: `/v3/users/5281843322b14bbab3715f6ac8531da5` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:56.928737 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e088a37-d526-46a4-9f56-220e28f71c13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5281843322b14bbab3715f6ac8531da5)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:56.930685 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e088a37-d526-46a4-9f56-220e28f71c13 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:57.219256 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-0e088a37-d526-46a4-9f56-220e28f71c13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5281843322b14bbab3715f6ac8531da5'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:57.219860 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-0e088a37-d526-46a4-9f56-220e28f71c13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5281843322b14bbab3715f6ac8531da5'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:57.231772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-0e088a37-d526-46a4-9f56-220e28f71c13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5281843322b14bbab3715f6ac8531da5'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:57.338104 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-0e088a37-d526-46a4-9f56-220e28f71c13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5281843322b14bbab3715f6ac8531da5'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:57.338739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-0e088a37-d526-46a4-9f56-220e28f71c13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5281843322b14bbab3715f6ac8531da5'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:57.352091 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1399/2776] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:05:56 2022] DELETE /identity/v3/users/5281843322b14bbab3715f6ac8531da5 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:05:57.825127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b1d31c-970e-48d8-bccc-43c88655c6a6 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:57.826691 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-37b1d31c-970e-48d8-bccc-43c88655c6a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:05:57.841058 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b1d31c-970e-48d8-bccc-43c88655c6a6 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:57.898967 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628bd2f-6306-4e1d-8458-5365d560f84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:57.899606 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628bd2f-6306-4e1d-8458-5365d560f84b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:57.899917 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628bd2f-6306-4e1d-8458-5365d560f84b admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:57.900191 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628bd2f-6306-4e1d-8458-5365d560f84b admin admin] PATH_INFO: `/v3/projects/f28bd8721a16436883c20dc85553bdc2` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:57.901363 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628bd2f-6306-4e1d-8458-5365d560f84b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f28bd8721a16436883c20dc85553bdc2)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:57.903447 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628bd2f-6306-4e1d-8458-5365d560f84b admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:58.014271 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-d628bd2f-6306-4e1d-8458-5365d560f84b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f28bd8721a16436883c20dc85553bdc2', 'request_id': 'req-d628bd2f-6306-4e1d-8458-5365d560f84b'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:58.080997 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-d628bd2f-6306-4e1d-8458-5365d560f84b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f28bd8721a16436883c20dc85553bdc2', 'request_id': 'req-d628bd2f-6306-4e1d-8458-5365d560f84b'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:58.087978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-d628bd2f-6306-4e1d-8458-5365d560f84b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f28bd8721a16436883c20dc85553bdc2', 'request_id': 'req-d628bd2f-6306-4e1d-8458-5365d560f84b'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:58.099172 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-d628bd2f-6306-4e1d-8458-5365d560f84b admin admin] The token cache is being invalidate because project f28bd8721a16436883c20dc85553bdc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:05:58.099457 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-d628bd2f-6306-4e1d-8458-5365d560f84b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:58.100118 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-d628bd2f-6306-4e1d-8458-5365d560f84b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:05:58.101635 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1378/2777] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:05:57 2022] DELETE /identity/v3/projects/f28bd8721a16436883c20dc85553bdc2 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:05:58.106557 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9d804-7fa0-4f2e-97d0-bb885b57a2d8 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:58.107033 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9d804-7fa0-4f2e-97d0-bb885b57a2d8 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:58.108349 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9d804-7fa0-4f2e-97d0-bb885b57a2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:58.127070 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-40a9d804-7fa0-4f2e-97d0-bb885b57a2d8 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:05:58.128262 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-40a9d804-7fa0-4f2e-97d0-bb885b57a2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:05:58.188048 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1400/2778] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:05:58.197303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02361e-c66a-4c37-a8d3-1129218c0114 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:58.197588 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02361e-c66a-4c37-a8d3-1129218c0114 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:58.197862 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02361e-c66a-4c37-a8d3-1129218c0114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:58.216904 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-ed02361e-c66a-4c37-a8d3-1129218c0114 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:05:58.218598 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-ed02361e-c66a-4c37-a8d3-1129218c0114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:05:58.226276 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1379/2779] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:05:58.230505 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9d804-7fa0-4f2e-97d0-bb885b57a2d8 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:58.240830 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9d804-7fa0-4f2e-97d0-bb885b57a2d8 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:58.245344 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b54303-33ea-4437-869c-2fbda92584d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:58.245940 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b54303-33ea-4437-869c-2fbda92584d2 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:58.246220 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b54303-33ea-4437-869c-2fbda92584d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:58.246490 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b54303-33ea-4437-869c-2fbda92584d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:58.247129 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b54303-33ea-4437-869c-2fbda92584d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 01:05:58.247434 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b54303-33ea-4437-869c-2fbda92584d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:58.249066 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b54303-33ea-4437-869c-2fbda92584d2 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:58.260640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1401/2780] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 01:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 01:05:58.264981 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02361e-c66a-4c37-a8d3-1129218c0114 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:58.274125 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02361e-c66a-4c37-a8d3-1129218c0114 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:58.278805 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9b05c-1a96-4a63-91f6-37491d9271a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:58.279880 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9b05c-1a96-4a63-91f6-37491d9271a6 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:58.280169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9b05c-1a96-4a63-91f6-37491d9271a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:58.280440 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9b05c-1a96-4a63-91f6-37491d9271a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:58.281090 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace9b05c-1a96-4a63-91f6-37491d9271a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:58.282999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace9b05c-1a96-4a63-91f6-37491d9271a6 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:58.408697 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1380/2781] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:05:58 2022] POST /identity/v3/projects => generated 401 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:05:58.413126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b54303-33ea-4437-869c-2fbda92584d2 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:58.424101 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b54303-33ea-4437-869c-2fbda92584d2 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:58.450003 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:05:58.480626 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959bb7d-d730-4b78-882d-0f30fb0125e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:58.481248 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959bb7d-d730-4b78-882d-0f30fb0125e6 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:58.481526 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959bb7d-d730-4b78-882d-0f30fb0125e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:58.481803 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959bb7d-d730-4b78-882d-0f30fb0125e6 admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:58.482432 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2959bb7d-d730-4b78-882d-0f30fb0125e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:58.484387 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2959bb7d-d730-4b78-882d-0f30fb0125e6 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:58.664183 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1402/2782] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:05:58 2022] POST /identity/v3/users => generated 301 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 01:05:58.668219 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9b05c-1a96-4a63-91f6-37491d9271a6 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:58.678924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9b05c-1a96-4a63-91f6-37491d9271a6 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:58.683475 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ff227-4937-4f37-b033-5a76ca87c5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:58.684077 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ff227-4937-4f37-b033-5a76ca87c5e8 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:58.684359 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ff227-4937-4f37-b033-5a76ca87c5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:58.684672 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ff227-4937-4f37-b033-5a76ca87c5e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:58.685338 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ff227-4937-4f37-b033-5a76ca87c5e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:58.687550 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ff227-4937-4f37-b033-5a76ca87c5e8 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:58.697582 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1381/2783] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:05:58.705911 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959bb7d-d730-4b78-882d-0f30fb0125e6 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:58.715640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959bb7d-d730-4b78-882d-0f30fb0125e6 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:58.720422 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e442443-f132-44d1-bee9-5bafb10ae813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:58.721056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e442443-f132-44d1-bee9-5bafb10ae813 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:58.721339 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e442443-f132-44d1-bee9-5bafb10ae813 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:58.721612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e442443-f132-44d1-bee9-5bafb10ae813 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:58.722256 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e442443-f132-44d1-bee9-5bafb10ae813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:58.723856 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e442443-f132-44d1-bee9-5bafb10ae813 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:58.734284 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1403/2784] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:05:58.738288 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ff227-4937-4f37-b033-5a76ca87c5e8 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:58.749196 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ff227-4937-4f37-b033-5a76ca87c5e8 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:58.753831 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6981c11-260f-4f77-badd-03a0a9c12750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:58.754486 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6981c11-260f-4f77-badd-03a0a9c12750 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:58.756464 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6981c11-260f-4f77-badd-03a0a9c12750 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:58.756464 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6981c11-260f-4f77-badd-03a0a9c12750 admin admin] PATH_INFO: `/v3/projects/a051efc3bd88452fbc6e904837089c9c/users/1b543a2a75d540dbb05806cc4946817c/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:58.769809 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6981c11-260f-4f77-badd-03a0a9c12750 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a051efc3bd88452fbc6e904837089c9c, user_id=1b543a2a75d540dbb05806cc4946817c, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:58.772520 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6981c11-260f-4f77-badd-03a0a9c12750 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:58.860075 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:05:58.864717 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1382/2785] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:05:58 2022] PUT /identity/v3/projects/a051efc3bd88452fbc6e904837089c9c/users/1b543a2a75d540dbb05806cc4946817c/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:05:58.875432 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e3d4b-8f7f-4602-9053-b6ec38fb8b52 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:58.875748 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e3d4b-8f7f-4602-9053-b6ec38fb8b52 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:58.876023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e3d4b-8f7f-4602-9053-b6ec38fb8b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:58.905410 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-b33e3d4b-8f7f-4602-9053-b6ec38fb8b52 None None] MFA Rules not processed for user `1b543a2a75d540dbb05806cc4946817c`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:05:58.906634 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-b33e3d4b-8f7f-4602-9053-b6ec38fb8b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:05:58.971719 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1404/2786] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:05:58 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 01:05:58.976083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6981c11-260f-4f77-badd-03a0a9c12750 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:59.038443 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6981c11-260f-4f77-badd-03a0a9c12750 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:59.043453 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72098eb1-eda7-44f1-b07f-9613df544d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:59.044065 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72098eb1-eda7-44f1-b07f-9613df544d76 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:59.044349 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72098eb1-eda7-44f1-b07f-9613df544d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:59.044660 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72098eb1-eda7-44f1-b07f-9613df544d76 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:59.045334 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72098eb1-eda7-44f1-b07f-9613df544d76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:59.047296 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72098eb1-eda7-44f1-b07f-9613df544d76 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:59.195746 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1383/2787] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:05:58 2022] POST /identity/v3/projects => generated 399 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:05:59.199806 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e3d4b-8f7f-4602-9053-b6ec38fb8b52 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:59.211628 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e3d4b-8f7f-4602-9053-b6ec38fb8b52 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:59.267089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2eaec-d6af-4438-aaba-a9a1b548e177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:59.267626 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2eaec-d6af-4438-aaba-a9a1b548e177 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:59.267915 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2eaec-d6af-4438-aaba-a9a1b548e177 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:59.268228 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2eaec-d6af-4438-aaba-a9a1b548e177 admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:59.268892 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb2eaec-d6af-4438-aaba-a9a1b548e177 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:59.270821 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb2eaec-d6af-4438-aaba-a9a1b548e177 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:59.361519 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1405/2788] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:05:59 2022] POST /identity/v3/users => generated 306 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:05:59.365322 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72098eb1-eda7-44f1-b07f-9613df544d76 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:59.380098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72098eb1-eda7-44f1-b07f-9613df544d76 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:59.384829 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750fc44-e72b-4187-b59a-603cf22cd4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:59.385434 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750fc44-e72b-4187-b59a-603cf22cd4d9 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:59.385714 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750fc44-e72b-4187-b59a-603cf22cd4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:59.385981 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750fc44-e72b-4187-b59a-603cf22cd4d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:59.386621 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6750fc44-e72b-4187-b59a-603cf22cd4d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:59.388207 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6750fc44-e72b-4187-b59a-603cf22cd4d9 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:59.399524 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1384/2789] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:05:59.403824 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2eaec-d6af-4438-aaba-a9a1b548e177 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:59.414986 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2eaec-d6af-4438-aaba-a9a1b548e177 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:59.419577 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ea202-7082-4c2e-81ed-6d5e04278196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:59.420377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ea202-7082-4c2e-81ed-6d5e04278196 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:59.420698 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ea202-7082-4c2e-81ed-6d5e04278196 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:59.420978 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ea202-7082-4c2e-81ed-6d5e04278196 admin admin] PATH_INFO: `/v3/projects/6db2af745acb4517a5284435c5b3587f/users/4f3c70d6f4ed428fa82013e7d5208a84/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:59.437103 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ea202-7082-4c2e-81ed-6d5e04278196 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6db2af745acb4517a5284435c5b3587f, user_id=4f3c70d6f4ed428fa82013e7d5208a84, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:59.439770 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ea202-7082-4c2e-81ed-6d5e04278196 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:59.504577 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1406/2790] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:05:59 2022] PUT /identity/v3/projects/6db2af745acb4517a5284435c5b3587f/users/4f3c70d6f4ed428fa82013e7d5208a84/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:05:59.508776 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750fc44-e72b-4187-b59a-603cf22cd4d9 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:59.519450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750fc44-e72b-4187-b59a-603cf22cd4d9 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:59.575361 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69361c72-1bc8-40aa-8425-6add37c5a18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:59.575963 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69361c72-1bc8-40aa-8425-6add37c5a18a admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:59.576258 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69361c72-1bc8-40aa-8425-6add37c5a18a admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:59.577080 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69361c72-1bc8-40aa-8425-6add37c5a18a admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:59.577726 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69361c72-1bc8-40aa-8425-6add37c5a18a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:59.579309 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69361c72-1bc8-40aa-8425-6add37c5a18a admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:59.588804 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1385/2791] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:05:59.593053 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ea202-7082-4c2e-81ed-6d5e04278196 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:59.598350 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:05:59.603550 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ea202-7082-4c2e-81ed-6d5e04278196 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:59.608373 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b77e9-97dc-4c97-9d2b-a1a0ed586e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:59.617459 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b77e9-97dc-4c97-9d2b-a1a0ed586e28 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:59.617459 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b77e9-97dc-4c97-9d2b-a1a0ed586e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:59.617459 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b77e9-97dc-4c97-9d2b-a1a0ed586e28 admin admin] PATH_INFO: `/v3/projects/6db2af745acb4517a5284435c5b3587f/users/4f3c70d6f4ed428fa82013e7d5208a84/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:59.617459 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b77e9-97dc-4c97-9d2b-a1a0ed586e28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6db2af745acb4517a5284435c5b3587f, user_id=4f3c70d6f4ed428fa82013e7d5208a84, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:59.617459 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b77e9-97dc-4c97-9d2b-a1a0ed586e28 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:59.624789 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1407/2792] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:05:59 2022] PUT /identity/v3/projects/6db2af745acb4517a5284435c5b3587f/users/4f3c70d6f4ed428fa82013e7d5208a84/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:05:59.635816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c50322-731e-48ab-8a29-b8c4688e5d42 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:59.636131 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c50322-731e-48ab-8a29-b8c4688e5d42 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:59.636413 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c50322-731e-48ab-8a29-b8c4688e5d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:59.665032 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-32c50322-731e-48ab-8a29-b8c4688e5d42 None None] MFA Rules not processed for user `4f3c70d6f4ed428fa82013e7d5208a84`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:05:59.666238 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-32c50322-731e-48ab-8a29-b8c4688e5d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:05:59.724621 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:05:59.733811 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1386/2793] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:05:59 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:05:59.744517 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b77e9-97dc-4c97-9d2b-a1a0ed586e28 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:05:59.746286 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-aa1b77e9-97dc-4c97-9d2b-a1a0ed586e28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:05:59.804458 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b77e9-97dc-4c97-9d2b-a1a0ed586e28 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:05:59.808142 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a3054-1c7b-406b-bcfc-0ce039428166 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:05:59.808785 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a3054-1c7b-406b-bcfc-0ce039428166 service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:05:59.809062 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a3054-1c7b-406b-bcfc-0ce039428166 service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:05:59.809332 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a3054-1c7b-406b-bcfc-0ce039428166 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:05:59.818767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a3054-1c7b-406b-bcfc-0ce039428166 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:05:59.820198 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a3054-1c7b-406b-bcfc-0ce039428166 service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:05:59.829549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1408/2794] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:05:59 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:06:00.066530 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c50322-731e-48ab-8a29-b8c4688e5d42 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:00.076294 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c50322-731e-48ab-8a29-b8c4688e5d42 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:00.079836 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85713954-ed77-4762-8734-a51b17b32630 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:00.080437 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85713954-ed77-4762-8734-a51b17b32630 service cinder] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:00.080750 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85713954-ed77-4762-8734-a51b17b32630 service cinder] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:00.081025 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85713954-ed77-4762-8734-a51b17b32630 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:00.138015 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85713954-ed77-4762-8734-a51b17b32630 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:00.139606 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85713954-ed77-4762-8734-a51b17b32630 service cinder] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:00.148961 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1387/2795] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:06:00 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:06:05.196114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a3054-1c7b-406b-bcfc-0ce039428166 service cinder] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:05.258614 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a3054-1c7b-406b-bcfc-0ce039428166 service cinder] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:05.263054 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63ad64-001a-4a82-9c42-c35d3cacf7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:05.264188 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63ad64-001a-4a82-9c42-c35d3cacf7a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:05.264466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63ad64-001a-4a82-9c42-c35d3cacf7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:05.264764 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63ad64-001a-4a82-9c42-c35d3cacf7a9 admin admin] PATH_INFO: `/v3/users/1b543a2a75d540dbb05806cc4946817c` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:05.265866 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63ad64-001a-4a82-9c42-c35d3cacf7a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b543a2a75d540dbb05806cc4946817c)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:05.267761 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63ad64-001a-4a82-9c42-c35d3cacf7a9 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:05.432318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2f63ad64-001a-4a82-9c42-c35d3cacf7a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b543a2a75d540dbb05806cc4946817c'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:05.432999 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2f63ad64-001a-4a82-9c42-c35d3cacf7a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b543a2a75d540dbb05806cc4946817c'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:05.445778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2f63ad64-001a-4a82-9c42-c35d3cacf7a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b543a2a75d540dbb05806cc4946817c'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:05.514901 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2f63ad64-001a-4a82-9c42-c35d3cacf7a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b543a2a75d540dbb05806cc4946817c'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:05.515506 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-2f63ad64-001a-4a82-9c42-c35d3cacf7a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b543a2a75d540dbb05806cc4946817c'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:05.527771 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1409/2796] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:06:05 2022] DELETE /identity/v3/users/1b543a2a75d540dbb05806cc4946817c => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:06:05.536214 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf0868-fd67-45ca-9548-ee6c53356ca0 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:05.536729 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf0868-fd67-45ca-9548-ee6c53356ca0 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:05.537145 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf0868-fd67-45ca-9548-ee6c53356ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:05.558001 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-baaf0868-fd67-45ca-9548-ee6c53356ca0 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:06:05.559761 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-baaf0868-fd67-45ca-9548-ee6c53356ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:05.617656 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1388/2797] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:06:05.627320 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63ad64-001a-4a82-9c42-c35d3cacf7a9 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:05.628744 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-2f63ad64-001a-4a82-9c42-c35d3cacf7a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:05.642116 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:06:05.686191 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63ad64-001a-4a82-9c42-c35d3cacf7a9 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:05.689770 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae263c0-b1dc-4f60-9300-894fb5f1458b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:05.690392 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae263c0-b1dc-4f60-9300-894fb5f1458b service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:05.690656 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae263c0-b1dc-4f60-9300-894fb5f1458b service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:05.690924 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae263c0-b1dc-4f60-9300-894fb5f1458b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:05.700811 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae263c0-b1dc-4f60-9300-894fb5f1458b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:05.702864 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae263c0-b1dc-4f60-9300-894fb5f1458b service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:05.712548 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1410/2798] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:06:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:06:06.122047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf0868-fd67-45ca-9548-ee6c53356ca0 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:06.123482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-baaf0868-fd67-45ca-9548-ee6c53356ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:06.135451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf0868-fd67-45ca-9548-ee6c53356ca0 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:06.140684 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea91189-4393-4b0b-8875-8a07f4bd0127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:06.141324 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea91189-4393-4b0b-8875-8a07f4bd0127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:06.141607 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea91189-4393-4b0b-8875-8a07f4bd0127 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:06.141884 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea91189-4393-4b0b-8875-8a07f4bd0127 admin admin] PATH_INFO: `/v3/projects/a051efc3bd88452fbc6e904837089c9c` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:06.143013 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea91189-4393-4b0b-8875-8a07f4bd0127 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a051efc3bd88452fbc6e904837089c9c)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:06.144961 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea91189-4393-4b0b-8875-8a07f4bd0127 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:06.261802 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-eea91189-4393-4b0b-8875-8a07f4bd0127 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a051efc3bd88452fbc6e904837089c9c', 'request_id': 'req-eea91189-4393-4b0b-8875-8a07f4bd0127'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:06.337756 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-eea91189-4393-4b0b-8875-8a07f4bd0127 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a051efc3bd88452fbc6e904837089c9c', 'request_id': 'req-eea91189-4393-4b0b-8875-8a07f4bd0127'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:06.345129 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-eea91189-4393-4b0b-8875-8a07f4bd0127 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a051efc3bd88452fbc6e904837089c9c', 'request_id': 'req-eea91189-4393-4b0b-8875-8a07f4bd0127'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:06.355845 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-eea91189-4393-4b0b-8875-8a07f4bd0127 admin admin] The token cache is being invalidate because project a051efc3bd88452fbc6e904837089c9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:06:06.356137 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-eea91189-4393-4b0b-8875-8a07f4bd0127 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:06.356835 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-eea91189-4393-4b0b-8875-8a07f4bd0127 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:06.359078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1389/2799] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:06:06 2022] DELETE /identity/v3/projects/a051efc3bd88452fbc6e904837089c9c => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:06:06.363078 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae263c0-b1dc-4f60-9300-894fb5f1458b service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:06.372767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae263c0-b1dc-4f60-9300-894fb5f1458b service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:06.432186 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1773c08-8e2a-48b9-a7a0-d76518c0101e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:06.432845 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1773c08-8e2a-48b9-a7a0-d76518c0101e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:06.433125 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1773c08-8e2a-48b9-a7a0-d76518c0101e admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:06.433397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1773c08-8e2a-48b9-a7a0-d76518c0101e admin admin] PATH_INFO: `/v3/users/4f3c70d6f4ed428fa82013e7d5208a84` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:06.434544 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1773c08-8e2a-48b9-a7a0-d76518c0101e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f3c70d6f4ed428fa82013e7d5208a84)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:06.436534 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1773c08-8e2a-48b9-a7a0-d76518c0101e admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:06.653572 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-c1773c08-8e2a-48b9-a7a0-d76518c0101e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f3c70d6f4ed428fa82013e7d5208a84'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:06.654733 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-c1773c08-8e2a-48b9-a7a0-d76518c0101e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f3c70d6f4ed428fa82013e7d5208a84'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:06.666954 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-c1773c08-8e2a-48b9-a7a0-d76518c0101e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f3c70d6f4ed428fa82013e7d5208a84'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:06.775944 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-c1773c08-8e2a-48b9-a7a0-d76518c0101e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f3c70d6f4ed428fa82013e7d5208a84'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:06.776628 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-c1773c08-8e2a-48b9-a7a0-d76518c0101e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f3c70d6f4ed428fa82013e7d5208a84'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:06.790151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1411/2800] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:06:06 2022] DELETE /identity/v3/users/4f3c70d6f4ed428fa82013e7d5208a84 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:06:07.287411 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea91189-4393-4b0b-8875-8a07f4bd0127 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:07.297511 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-eea91189-4393-4b0b-8875-8a07f4bd0127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:07.304467 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea91189-4393-4b0b-8875-8a07f4bd0127 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:07.317870 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:06:07.362662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01adb92-3459-4cf4-a21f-8f1620552480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:07.364670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01adb92-3459-4cf4-a21f-8f1620552480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:07.364670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01adb92-3459-4cf4-a21f-8f1620552480 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:07.364670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01adb92-3459-4cf4-a21f-8f1620552480 admin admin] PATH_INFO: `/v3/projects/6db2af745acb4517a5284435c5b3587f` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:07.365047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01adb92-3459-4cf4-a21f-8f1620552480 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6db2af745acb4517a5284435c5b3587f)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:07.366946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01adb92-3459-4cf4-a21f-8f1620552480 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:07.515908 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-a01adb92-3459-4cf4-a21f-8f1620552480 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6db2af745acb4517a5284435c5b3587f', 'request_id': 'req-a01adb92-3459-4cf4-a21f-8f1620552480'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:07.646208 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-a01adb92-3459-4cf4-a21f-8f1620552480 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6db2af745acb4517a5284435c5b3587f', 'request_id': 'req-a01adb92-3459-4cf4-a21f-8f1620552480'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:07.652694 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-a01adb92-3459-4cf4-a21f-8f1620552480 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6db2af745acb4517a5284435c5b3587f', 'request_id': 'req-a01adb92-3459-4cf4-a21f-8f1620552480'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:07.663702 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-a01adb92-3459-4cf4-a21f-8f1620552480 admin admin] The token cache is being invalidate because project 6db2af745acb4517a5284435c5b3587f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:06:07.663995 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-a01adb92-3459-4cf4-a21f-8f1620552480 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:07.664668 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-a01adb92-3459-4cf4-a21f-8f1620552480 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:07.666147 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1390/2801] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:06:07 2022] DELETE /identity/v3/projects/6db2af745acb4517a5284435c5b3587f => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:06:07.670983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1479fe-2970-4afa-8486-3a26b6447aa6 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:07.671459 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1479fe-2970-4afa-8486-3a26b6447aa6 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:07.671590 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1479fe-2970-4afa-8486-3a26b6447aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:07.690955 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-6e1479fe-2970-4afa-8486-3a26b6447aa6 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:06:07.692155 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-6e1479fe-2970-4afa-8486-3a26b6447aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:07.722146 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:06:07.750789 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1412/2802] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:06:07.758593 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab92b7-11a7-4afe-8576-b1e2a3d9b142 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:07.758882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab92b7-11a7-4afe-8576-b1e2a3d9b142 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:07.759171 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab92b7-11a7-4afe-8576-b1e2a3d9b142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:07.780478 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-adab92b7-11a7-4afe-8576-b1e2a3d9b142 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:06:07.781699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-adab92b7-11a7-4afe-8576-b1e2a3d9b142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:07.789856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1391/2803] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:06:07.794233 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1479fe-2970-4afa-8486-3a26b6447aa6 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:07.804331 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1479fe-2970-4afa-8486-3a26b6447aa6 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:07.808863 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bb8a9-472a-4939-b698-73fe8cec94ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:07.809492 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bb8a9-472a-4939-b698-73fe8cec94ce admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:07.809778 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bb8a9-472a-4939-b698-73fe8cec94ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:07.810052 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bb8a9-472a-4939-b698-73fe8cec94ce admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:07.811192 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bb8a9-472a-4939-b698-73fe8cec94ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 01:06:07.811491 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bb8a9-472a-4939-b698-73fe8cec94ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:07.813121 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bb8a9-472a-4939-b698-73fe8cec94ce admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:07.824789 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1413/2804] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 01:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 01:06:07.829311 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab92b7-11a7-4afe-8576-b1e2a3d9b142 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:07.838468 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab92b7-11a7-4afe-8576-b1e2a3d9b142 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:07.843298 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e26a5-407b-44fe-be6d-0e3d8eee149a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:07.843891 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e26a5-407b-44fe-be6d-0e3d8eee149a admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:07.844208 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e26a5-407b-44fe-be6d-0e3d8eee149a admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:07.844442 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e26a5-407b-44fe-be6d-0e3d8eee149a admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:07.845100 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e26a5-407b-44fe-be6d-0e3d8eee149a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:07.846998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e26a5-407b-44fe-be6d-0e3d8eee149a admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:07.968414 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1392/2805] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:06:07 2022] POST /identity/v3/projects => generated 387 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:06:07.972461 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bb8a9-472a-4939-b698-73fe8cec94ce admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:07.982628 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bb8a9-472a-4939-b698-73fe8cec94ce admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:08.040324 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb81900-ce61-423b-87dc-ef84ccea6d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:08.040965 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb81900-ce61-423b-87dc-ef84ccea6d47 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:08.041249 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb81900-ce61-423b-87dc-ef84ccea6d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:08.041545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb81900-ce61-423b-87dc-ef84ccea6d47 admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:08.042161 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb81900-ce61-423b-87dc-ef84ccea6d47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:08.044089 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb81900-ce61-423b-87dc-ef84ccea6d47 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:08.179088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1414/2806] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:06:07 2022] POST /identity/v3/users => generated 294 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 01:06:08.193882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e26a5-407b-44fe-be6d-0e3d8eee149a admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:08.204841 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e26a5-407b-44fe-be6d-0e3d8eee149a admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:08.209303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4d42c-dc1b-4a3d-87a7-ce81fbdb6676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:08.209906 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4d42c-dc1b-4a3d-87a7-ce81fbdb6676 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:08.210189 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4d42c-dc1b-4a3d-87a7-ce81fbdb6676 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:08.210464 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4d42c-dc1b-4a3d-87a7-ce81fbdb6676 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:08.211108 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df4d42c-dc1b-4a3d-87a7-ce81fbdb6676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:08.212733 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df4d42c-dc1b-4a3d-87a7-ce81fbdb6676 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:08.223056 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1393/2807] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:06:08.227126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb81900-ce61-423b-87dc-ef84ccea6d47 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:08.237015 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb81900-ce61-423b-87dc-ef84ccea6d47 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:08.241545 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f59471-14d2-41a3-81f0-173f8018d9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:08.242136 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f59471-14d2-41a3-81f0-173f8018d9b9 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:08.242414 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f59471-14d2-41a3-81f0-173f8018d9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:08.242684 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f59471-14d2-41a3-81f0-173f8018d9b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:08.243320 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f59471-14d2-41a3-81f0-173f8018d9b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:08.244947 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f59471-14d2-41a3-81f0-173f8018d9b9 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:08.255220 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1415/2808] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:06:08.259267 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4d42c-dc1b-4a3d-87a7-ce81fbdb6676 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:08.269578 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4d42c-dc1b-4a3d-87a7-ce81fbdb6676 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:08.274113 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7fa82a-8158-4ed7-928f-ddfe28078605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:08.274755 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fa82a-8158-4ed7-928f-ddfe28078605 admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:08.275036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fa82a-8158-4ed7-928f-ddfe28078605 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:08.275312 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fa82a-8158-4ed7-928f-ddfe28078605 admin admin] PATH_INFO: `/v3/projects/76d6a05ee02541b0a362a4e65a4368c5/users/ae6e2e0dc0cb406e90659beebe3daa4c/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:08.290407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7fa82a-8158-4ed7-928f-ddfe28078605 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d6a05ee02541b0a362a4e65a4368c5, user_id=ae6e2e0dc0cb406e90659beebe3daa4c, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:08.293099 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7fa82a-8158-4ed7-928f-ddfe28078605 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:08.413363 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1394/2809] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:06:08 2022] PUT /identity/v3/projects/76d6a05ee02541b0a362a4e65a4368c5/users/ae6e2e0dc0cb406e90659beebe3daa4c/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 01:06:08.423638 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579bfd53-25a4-4dee-8cda-1ded83e169d1 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:08.423927 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579bfd53-25a4-4dee-8cda-1ded83e169d1 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:08.424206 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579bfd53-25a4-4dee-8cda-1ded83e169d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:08.453631 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-579bfd53-25a4-4dee-8cda-1ded83e169d1 None None] MFA Rules not processed for user `ae6e2e0dc0cb406e90659beebe3daa4c`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:06:08.454929 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-579bfd53-25a4-4dee-8cda-1ded83e169d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:08.518036 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1416/2810] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:06:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:06:08.522349 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7fa82a-8158-4ed7-928f-ddfe28078605 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:08.567127 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:06:08.584374 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7fa82a-8158-4ed7-928f-ddfe28078605 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:08.589047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e2965-e508-42a6-b775-3c9f91d5c60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:08.589649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e2965-e508-42a6-b775-3c9f91d5c60e admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:08.589929 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e2965-e508-42a6-b775-3c9f91d5c60e admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:08.590204 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e2965-e508-42a6-b775-3c9f91d5c60e admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:08.590863 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e2965-e508-42a6-b775-3c9f91d5c60e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:08.592923 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e2965-e508-42a6-b775-3c9f91d5c60e admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:08.687424 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1395/2811] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:06:08 2022] POST /identity/v3/projects => generated 385 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 01:06:08.691642 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579bfd53-25a4-4dee-8cda-1ded83e169d1 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:08.702896 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579bfd53-25a4-4dee-8cda-1ded83e169d1 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:08.729370 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:06:08.760067 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfae12b-0284-4e0f-b347-2f7b60da92d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:08.760719 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfae12b-0284-4e0f-b347-2f7b60da92d0 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:08.761000 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfae12b-0284-4e0f-b347-2f7b60da92d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:08.761271 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfae12b-0284-4e0f-b347-2f7b60da92d0 admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:08.761903 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfae12b-0284-4e0f-b347-2f7b60da92d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:08.764027 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfae12b-0284-4e0f-b347-2f7b60da92d0 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:08.927316 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1417/2812] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:06:08 2022] POST /identity/v3/users => generated 299 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 01:06:08.931158 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e2965-e508-42a6-b775-3c9f91d5c60e admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:08.941595 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e2965-e508-42a6-b775-3c9f91d5c60e admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:08.946308 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f62dec-914f-4885-a8e8-42568d360e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:08.946917 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f62dec-914f-4885-a8e8-42568d360e7b admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:08.947201 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f62dec-914f-4885-a8e8-42568d360e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:08.947474 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f62dec-914f-4885-a8e8-42568d360e7b admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:08.948180 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f62dec-914f-4885-a8e8-42568d360e7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:08.949873 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f62dec-914f-4885-a8e8-42568d360e7b admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:08.960781 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1396/2813] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:06:08.965344 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfae12b-0284-4e0f-b347-2f7b60da92d0 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:08.976007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfae12b-0284-4e0f-b347-2f7b60da92d0 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:08.980872 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c1c5ad-9b0f-406b-8599-a4b7a94ca206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:08.981462 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c1c5ad-9b0f-406b-8599-a4b7a94ca206 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:08.981746 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c1c5ad-9b0f-406b-8599-a4b7a94ca206 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:08.982023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c1c5ad-9b0f-406b-8599-a4b7a94ca206 admin admin] PATH_INFO: `/v3/projects/b47028f18f3a4af39e5aa7f10d3b54a8/users/cdb9f003937f4d92a0156ae0bdb80767/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:08.996735 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c1c5ad-9b0f-406b-8599-a4b7a94ca206 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b47028f18f3a4af39e5aa7f10d3b54a8, user_id=cdb9f003937f4d92a0156ae0bdb80767, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:08.999502 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c1c5ad-9b0f-406b-8599-a4b7a94ca206 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:09.104236 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1418/2814] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:06:08 2022] PUT /identity/v3/projects/b47028f18f3a4af39e5aa7f10d3b54a8/users/cdb9f003937f4d92a0156ae0bdb80767/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:06:09.108283 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f62dec-914f-4885-a8e8-42568d360e7b admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:09.118124 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f62dec-914f-4885-a8e8-42568d360e7b admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:09.172670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362dffa-7f12-4040-8237-679ab57085fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:09.173279 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362dffa-7f12-4040-8237-679ab57085fc admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:09.173562 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362dffa-7f12-4040-8237-679ab57085fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:09.173836 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362dffa-7f12-4040-8237-679ab57085fc admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:09.174472 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4362dffa-7f12-4040-8237-679ab57085fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:09.176065 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4362dffa-7f12-4040-8237-679ab57085fc admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:09.185550 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1397/2815] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:06:09.189846 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c1c5ad-9b0f-406b-8599-a4b7a94ca206 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:09.199949 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c1c5ad-9b0f-406b-8599-a4b7a94ca206 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:09.205318 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff800b-d50e-4326-acda-edb2a45c4fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:09.205971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff800b-d50e-4326-acda-edb2a45c4fbe admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:09.206259 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff800b-d50e-4326-acda-edb2a45c4fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:09.206532 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff800b-d50e-4326-acda-edb2a45c4fbe admin admin] PATH_INFO: `/v3/projects/b47028f18f3a4af39e5aa7f10d3b54a8/users/cdb9f003937f4d92a0156ae0bdb80767/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:09.208447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff800b-d50e-4326-acda-edb2a45c4fbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=b47028f18f3a4af39e5aa7f10d3b54a8, user_id=cdb9f003937f4d92a0156ae0bdb80767, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:09.211214 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff800b-d50e-4326-acda-edb2a45c4fbe admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:09.221976 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1419/2816] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:06:09 2022] PUT /identity/v3/projects/b47028f18f3a4af39e5aa7f10d3b54a8/users/cdb9f003937f4d92a0156ae0bdb80767/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:06:09.286243 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55f1ac-2971-4dfd-9e4f-c803f5bdb043 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:09.286459 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55f1ac-2971-4dfd-9e4f-c803f5bdb043 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:09.286704 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55f1ac-2971-4dfd-9e4f-c803f5bdb043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:09.315165 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-4d55f1ac-2971-4dfd-9e4f-c803f5bdb043 None None] MFA Rules not processed for user `cdb9f003937f4d92a0156ae0bdb80767`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:06:09.316352 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-4d55f1ac-2971-4dfd-9e4f-c803f5bdb043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:09.383509 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1398/2817] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:06:09 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:06:09.394221 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff800b-d50e-4326-acda-edb2a45c4fbe admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:09.395926 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-51ff800b-d50e-4326-acda-edb2a45c4fbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:09.454747 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff800b-d50e-4326-acda-edb2a45c4fbe admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:09.458388 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3244803-9d07-4553-bf78-ca335ea00927 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:09.458997 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3244803-9d07-4553-bf78-ca335ea00927 service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:09.459278 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3244803-9d07-4553-bf78-ca335ea00927 service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:09.459554 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3244803-9d07-4553-bf78-ca335ea00927 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:09.469166 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3244803-9d07-4553-bf78-ca335ea00927 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:09.470612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3244803-9d07-4553-bf78-ca335ea00927 service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:09.480079 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1420/2818] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:06:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:06:11.117162 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55f1ac-2971-4dfd-9e4f-c803f5bdb043 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:11.159079 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:06:11.179036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55f1ac-2971-4dfd-9e4f-c803f5bdb043 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:11.183603 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7eb37b-6dac-4dc6-b67a-29f85d8a7c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:11.184247 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7eb37b-6dac-4dc6-b67a-29f85d8a7c33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:11.184529 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7eb37b-6dac-4dc6-b67a-29f85d8a7c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:11.184807 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7eb37b-6dac-4dc6-b67a-29f85d8a7c33 admin admin] PATH_INFO: `/v3/users/ae6e2e0dc0cb406e90659beebe3daa4c` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:11.185973 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7eb37b-6dac-4dc6-b67a-29f85d8a7c33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae6e2e0dc0cb406e90659beebe3daa4c)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:11.187899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7eb37b-6dac-4dc6-b67a-29f85d8a7c33 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:11.481994 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0f7eb37b-6dac-4dc6-b67a-29f85d8a7c33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae6e2e0dc0cb406e90659beebe3daa4c'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:11.497302 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0f7eb37b-6dac-4dc6-b67a-29f85d8a7c33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae6e2e0dc0cb406e90659beebe3daa4c'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:11.566152 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0f7eb37b-6dac-4dc6-b67a-29f85d8a7c33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae6e2e0dc0cb406e90659beebe3daa4c'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:11.566786 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0f7eb37b-6dac-4dc6-b67a-29f85d8a7c33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae6e2e0dc0cb406e90659beebe3daa4c'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:11.567394 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0f7eb37b-6dac-4dc6-b67a-29f85d8a7c33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae6e2e0dc0cb406e90659beebe3daa4c'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:11.580580 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1399/2819] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:06:11 2022] DELETE /identity/v3/users/ae6e2e0dc0cb406e90659beebe3daa4c => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:06:11.585082 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b130e-ef23-4a8b-b2fd-ac22b92f93c9 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:11.585613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b130e-ef23-4a8b-b2fd-ac22b92f93c9 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:11.585613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b130e-ef23-4a8b-b2fd-ac22b92f93c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:11.604796 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-873b130e-ef23-4a8b-b2fd-ac22b92f93c9 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:06:11.606030 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-873b130e-ef23-4a8b-b2fd-ac22b92f93c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:11.664444 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1421/2820] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:06:11.674924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7eb37b-6dac-4dc6-b67a-29f85d8a7c33 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:11.676362 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-0f7eb37b-6dac-4dc6-b67a-29f85d8a7c33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:11.732032 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7eb37b-6dac-4dc6-b67a-29f85d8a7c33 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:11.736202 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f7ce9-39bc-4173-9f75-af842fc3f20a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:11.736829 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f7ce9-39bc-4173-9f75-af842fc3f20a service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:11.737115 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f7ce9-39bc-4173-9f75-af842fc3f20a service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:11.737388 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f7ce9-39bc-4173-9f75-af842fc3f20a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:11.746176 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f7ce9-39bc-4173-9f75-af842fc3f20a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:11.747606 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f7ce9-39bc-4173-9f75-af842fc3f20a service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:11.757028 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1400/2821] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:06:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:06:12.329030 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b130e-ef23-4a8b-b2fd-ac22b92f93c9 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:12.329030 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-873b130e-ef23-4a8b-b2fd-ac22b92f93c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:12.340184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b130e-ef23-4a8b-b2fd-ac22b92f93c9 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:12.345366 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b3588b-188d-4c8b-8f0f-406aa291aa29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:12.346008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3588b-188d-4c8b-8f0f-406aa291aa29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:12.346287 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3588b-188d-4c8b-8f0f-406aa291aa29 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:12.346564 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3588b-188d-4c8b-8f0f-406aa291aa29 admin admin] PATH_INFO: `/v3/projects/76d6a05ee02541b0a362a4e65a4368c5` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:12.347681 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b3588b-188d-4c8b-8f0f-406aa291aa29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76d6a05ee02541b0a362a4e65a4368c5)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:12.349707 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b3588b-188d-4c8b-8f0f-406aa291aa29 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:12.371634 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:06:12.562026 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-35b3588b-188d-4c8b-8f0f-406aa291aa29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76d6a05ee02541b0a362a4e65a4368c5', 'request_id': 'req-35b3588b-188d-4c8b-8f0f-406aa291aa29'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:12.690613 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-35b3588b-188d-4c8b-8f0f-406aa291aa29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76d6a05ee02541b0a362a4e65a4368c5', 'request_id': 'req-35b3588b-188d-4c8b-8f0f-406aa291aa29'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:12.696327 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-35b3588b-188d-4c8b-8f0f-406aa291aa29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76d6a05ee02541b0a362a4e65a4368c5', 'request_id': 'req-35b3588b-188d-4c8b-8f0f-406aa291aa29'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:12.708024 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-35b3588b-188d-4c8b-8f0f-406aa291aa29 admin admin] The token cache is being invalidate because project 76d6a05ee02541b0a362a4e65a4368c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:06:12.708534 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-35b3588b-188d-4c8b-8f0f-406aa291aa29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:12.709598 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-35b3588b-188d-4c8b-8f0f-406aa291aa29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:12.711076 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1422/2822] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:06:12 2022] DELETE /identity/v3/projects/76d6a05ee02541b0a362a4e65a4368c5 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:06:12.715101 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f7ce9-39bc-4173-9f75-af842fc3f20a service neutron] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:12.725500 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f7ce9-39bc-4173-9f75-af842fc3f20a service neutron] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:12.780864 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a84e0-6573-46be-a879-ba144201250b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:12.781498 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a84e0-6573-46be-a879-ba144201250b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:12.781780 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a84e0-6573-46be-a879-ba144201250b admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:12.782053 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a84e0-6573-46be-a879-ba144201250b admin admin] PATH_INFO: `/v3/users/cdb9f003937f4d92a0156ae0bdb80767` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:12.783182 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a84e0-6573-46be-a879-ba144201250b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdb9f003937f4d92a0156ae0bdb80767)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:12.785168 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a84e0-6573-46be-a879-ba144201250b admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:12.787817 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:06:13.226610 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-323a84e0-6573-46be-a879-ba144201250b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdb9f003937f4d92a0156ae0bdb80767'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:13.247285 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-323a84e0-6573-46be-a879-ba144201250b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdb9f003937f4d92a0156ae0bdb80767'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:13.373438 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-323a84e0-6573-46be-a879-ba144201250b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdb9f003937f4d92a0156ae0bdb80767'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:13.374074 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-323a84e0-6573-46be-a879-ba144201250b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdb9f003937f4d92a0156ae0bdb80767'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:13.374678 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-323a84e0-6573-46be-a879-ba144201250b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdb9f003937f4d92a0156ae0bdb80767'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:13.387357 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1401/2823] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:06:12 2022] DELETE /identity/v3/users/cdb9f003937f4d92a0156ae0bdb80767 => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:06:13.991154 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b3588b-188d-4c8b-8f0f-406aa291aa29 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:13.992694 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-35b3588b-188d-4c8b-8f0f-406aa291aa29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:14.008276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b3588b-188d-4c8b-8f0f-406aa291aa29 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:14.065642 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a28277-cd14-4252-a1f7-fd89b210a3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:14.066276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a28277-cd14-4252-a1f7-fd89b210a3fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:14.066562 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a28277-cd14-4252-a1f7-fd89b210a3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:14.066840 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a28277-cd14-4252-a1f7-fd89b210a3fb admin admin] PATH_INFO: `/v3/projects/b47028f18f3a4af39e5aa7f10d3b54a8` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:14.067971 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a28277-cd14-4252-a1f7-fd89b210a3fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b47028f18f3a4af39e5aa7f10d3b54a8)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:14.069915 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a28277-cd14-4252-a1f7-fd89b210a3fb admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:14.234230 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-24a28277-cd14-4252-a1f7-fd89b210a3fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b47028f18f3a4af39e5aa7f10d3b54a8', 'request_id': 'req-24a28277-cd14-4252-a1f7-fd89b210a3fb'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:14.326980 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-24a28277-cd14-4252-a1f7-fd89b210a3fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b47028f18f3a4af39e5aa7f10d3b54a8', 'request_id': 'req-24a28277-cd14-4252-a1f7-fd89b210a3fb'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:14.342404 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-24a28277-cd14-4252-a1f7-fd89b210a3fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b47028f18f3a4af39e5aa7f10d3b54a8', 'request_id': 'req-24a28277-cd14-4252-a1f7-fd89b210a3fb'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:14.345120 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-24a28277-cd14-4252-a1f7-fd89b210a3fb admin admin] The token cache is being invalidate because project b47028f18f3a4af39e5aa7f10d3b54a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:06:14.345415 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-24a28277-cd14-4252-a1f7-fd89b210a3fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:14.346050 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-24a28277-cd14-4252-a1f7-fd89b210a3fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:14.347539 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1423/2824] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:06:13 2022] DELETE /identity/v3/projects/b47028f18f3a4af39e5aa7f10d3b54a8 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:06:14.352462 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601827ba-e8d8-4f2b-8060-4531d59944c6 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:14.352970 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601827ba-e8d8-4f2b-8060-4531d59944c6 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:14.353073 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601827ba-e8d8-4f2b-8060-4531d59944c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:14.372828 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-601827ba-e8d8-4f2b-8060-4531d59944c6 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:06:14.374048 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-601827ba-e8d8-4f2b-8060-4531d59944c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:14.432958 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1402/2825] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:06:14.441146 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607785f-365b-467b-a1a1-79e1754f896e None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:14.441565 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607785f-365b-467b-a1a1-79e1754f896e None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:14.441850 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607785f-365b-467b-a1a1-79e1754f896e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:14.462406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-b607785f-365b-467b-a1a1-79e1754f896e None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:06:14.463624 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-b607785f-365b-467b-a1a1-79e1754f896e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:14.472099 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1424/2826] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:06:14.476397 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601827ba-e8d8-4f2b-8060-4531d59944c6 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:14.486484 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601827ba-e8d8-4f2b-8060-4531d59944c6 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:14.491082 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6495e5-5ef9-4476-8735-36b768e5c144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:14.491688 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6495e5-5ef9-4476-8735-36b768e5c144 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:14.491974 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6495e5-5ef9-4476-8735-36b768e5c144 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:14.492245 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6495e5-5ef9-4476-8735-36b768e5c144 admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:14.492944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6495e5-5ef9-4476-8735-36b768e5c144 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 01:06:14.493253 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6495e5-5ef9-4476-8735-36b768e5c144 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:14.495001 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6495e5-5ef9-4476-8735-36b768e5c144 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:14.506430 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1403/2827] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 01:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 01:06:14.510581 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b607785f-365b-467b-a1a1-79e1754f896e None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:14.520397 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b607785f-365b-467b-a1a1-79e1754f896e None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:14.525213 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843881d6-eaba-404f-b5ec-d3466ba23ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:14.525816 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843881d6-eaba-404f-b5ec-d3466ba23ace admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:14.526099 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843881d6-eaba-404f-b5ec-d3466ba23ace admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:14.526374 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843881d6-eaba-404f-b5ec-d3466ba23ace admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:14.527007 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843881d6-eaba-404f-b5ec-d3466ba23ace admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:14.529527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843881d6-eaba-404f-b5ec-d3466ba23ace admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:14.708124 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1425/2828] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:06:14 2022] POST /identity/v3/projects => generated 377 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:06:14.709151 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6495e5-5ef9-4476-8735-36b768e5c144 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:14.719540 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6495e5-5ef9-4476-8735-36b768e5c144 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:14.774909 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b3e70-4f77-4c33-b40d-074bea069075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:14.775517 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b3e70-4f77-4c33-b40d-074bea069075 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:14.775798 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b3e70-4f77-4c33-b40d-074bea069075 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:14.776072 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b3e70-4f77-4c33-b40d-074bea069075 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:14.776804 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b3e70-4f77-4c33-b40d-074bea069075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:14.778728 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b3e70-4f77-4c33-b40d-074bea069075 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:14.886940 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:06:14.893184 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1404/2829] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:06:14 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:06:14.897292 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843881d6-eaba-404f-b5ec-d3466ba23ace admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:14.907773 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843881d6-eaba-404f-b5ec-d3466ba23ace admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:14.912628 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f01da0-ace0-4b40-b100-82038d460bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:14.913234 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f01da0-ace0-4b40-b100-82038d460bca admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:14.913514 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f01da0-ace0-4b40-b100-82038d460bca admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:14.913831 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f01da0-ace0-4b40-b100-82038d460bca admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:14.914470 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f01da0-ace0-4b40-b100-82038d460bca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:14.916060 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f01da0-ace0-4b40-b100-82038d460bca admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:14.918178 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:06:14.927072 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1426/2830] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:06:14.930946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b3e70-4f77-4c33-b40d-074bea069075 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:14.942569 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b3e70-4f77-4c33-b40d-074bea069075 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:14.947220 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a69d68-58cc-49e8-8524-aed9a734cf4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:14.947825 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a69d68-58cc-49e8-8524-aed9a734cf4e admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:14.948107 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a69d68-58cc-49e8-8524-aed9a734cf4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:14.948388 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a69d68-58cc-49e8-8524-aed9a734cf4e admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:14.949076 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a69d68-58cc-49e8-8524-aed9a734cf4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:14.950705 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a69d68-58cc-49e8-8524-aed9a734cf4e admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:14.965343 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1405/2831] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:06:14.965823 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f01da0-ace0-4b40-b100-82038d460bca admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:14.976616 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f01da0-ace0-4b40-b100-82038d460bca admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:14.981220 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00369f-7691-4517-96b2-ef239237a52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:14.981893 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00369f-7691-4517-96b2-ef239237a52c admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:14.982723 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00369f-7691-4517-96b2-ef239237a52c admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:14.982945 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00369f-7691-4517-96b2-ef239237a52c admin admin] PATH_INFO: `/v3/projects/2ab9a532d467403aa288e3040406dd96/users/18a70915e80b417d9dfa786781c18066/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:14.997381 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00369f-7691-4517-96b2-ef239237a52c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ab9a532d467403aa288e3040406dd96, user_id=18a70915e80b417d9dfa786781c18066, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:15.000658 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00369f-7691-4517-96b2-ef239237a52c admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:15.069063 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1427/2832] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:06:14 2022] PUT /identity/v3/projects/2ab9a532d467403aa288e3040406dd96/users/18a70915e80b417d9dfa786781c18066/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:06:15.079419 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd245e3e-30d9-4807-a301-0d0dff57ff68 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:15.079708 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd245e3e-30d9-4807-a301-0d0dff57ff68 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:15.079988 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd245e3e-30d9-4807-a301-0d0dff57ff68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:15.110464 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-cd245e3e-30d9-4807-a301-0d0dff57ff68 None None] MFA Rules not processed for user `18a70915e80b417d9dfa786781c18066`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:06:15.111710 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-cd245e3e-30d9-4807-a301-0d0dff57ff68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:15.180457 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1406/2833] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:06:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:06:15.192694 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00369f-7691-4517-96b2-ef239237a52c admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:15.194354 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-1c00369f-7691-4517-96b2-ef239237a52c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:15.270220 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00369f-7691-4517-96b2-ef239237a52c admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:15.273962 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f07e3-baf4-4ea6-9a09-367e212ccc8e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abed7ebb0aae4e4693ce08e51a149312', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:15.274571 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f07e3-baf4-4ea6-9a09-367e212ccc8e service glance] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:15.274853 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f07e3-baf4-4ea6-9a09-367e212ccc8e service glance] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:15.275150 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f07e3-baf4-4ea6-9a09-367e212ccc8e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:15.283521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f07e3-baf4-4ea6-9a09-367e212ccc8e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:15.285006 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f07e3-baf4-4ea6-9a09-367e212ccc8e service glance] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:15.293973 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1428/2834] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:06:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 01:06:25.177542 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=38 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:06:25.290053 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=46,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:06:33.509417 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd245e3e-30d9-4807-a301-0d0dff57ff68 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:33.576914 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd245e3e-30d9-4807-a301-0d0dff57ff68 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:33.581499 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f3a26-7bcc-4005-8b16-d2fa32d876c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:33.582132 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f3a26-7bcc-4005-8b16-d2fa32d876c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:33.582411 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f3a26-7bcc-4005-8b16-d2fa32d876c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:33.582748 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f3a26-7bcc-4005-8b16-d2fa32d876c9 admin admin] PATH_INFO: `/v3/users/18a70915e80b417d9dfa786781c18066` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:33.583873 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f3a26-7bcc-4005-8b16-d2fa32d876c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18a70915e80b417d9dfa786781c18066)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:33.585810 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f3a26-7bcc-4005-8b16-d2fa32d876c9 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:33.785893 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-661f3a26-7bcc-4005-8b16-d2fa32d876c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18a70915e80b417d9dfa786781c18066'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:33.801394 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-661f3a26-7bcc-4005-8b16-d2fa32d876c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18a70915e80b417d9dfa786781c18066'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:33.863247 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-661f3a26-7bcc-4005-8b16-d2fa32d876c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18a70915e80b417d9dfa786781c18066'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:33.863942 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-661f3a26-7bcc-4005-8b16-d2fa32d876c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18a70915e80b417d9dfa786781c18066'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:33.864662 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-661f3a26-7bcc-4005-8b16-d2fa32d876c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18a70915e80b417d9dfa786781c18066'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:33.879262 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1407/2835] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:06:33 2022] DELETE /identity/v3/users/18a70915e80b417d9dfa786781c18066 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:06:33.883705 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3bccf1-09e4-418e-b1f5-6202b23f386c None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:33.884218 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3bccf1-09e4-418e-b1f5-6202b23f386c None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:33.884335 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3bccf1-09e4-418e-b1f5-6202b23f386c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:33.905008 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-9c3bccf1-09e4-418e-b1f5-6202b23f386c None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:06:33.906243 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-9c3bccf1-09e4-418e-b1f5-6202b23f386c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:33.998023 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1429/2836] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:06:34.005005 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f3a26-7bcc-4005-8b16-d2fa32d876c9 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:34.006564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-661f3a26-7bcc-4005-8b16-d2fa32d876c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:34.081358 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f3a26-7bcc-4005-8b16-d2fa32d876c9 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:34.085656 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284ea78-0235-4376-9abd-5814a797f68a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:34.086376 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284ea78-0235-4376-9abd-5814a797f68a service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:34.086723 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284ea78-0235-4376-9abd-5814a797f68a service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:34.087062 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284ea78-0235-4376-9abd-5814a797f68a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:34.099940 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284ea78-0235-4376-9abd-5814a797f68a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:34.101557 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284ea78-0235-4376-9abd-5814a797f68a service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:34.113435 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1408/2837] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:06:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:06:34.798533 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3bccf1-09e4-418e-b1f5-6202b23f386c None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:34.800377 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-9c3bccf1-09e4-418e-b1f5-6202b23f386c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:34.818122 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3bccf1-09e4-418e-b1f5-6202b23f386c None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:34.823315 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702022a3-17f9-463d-b192-98c260cd9c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:34.823592 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702022a3-17f9-463d-b192-98c260cd9c81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:34.823701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702022a3-17f9-463d-b192-98c260cd9c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:34.823953 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702022a3-17f9-463d-b192-98c260cd9c81 admin admin] PATH_INFO: `/v3/projects/2ab9a532d467403aa288e3040406dd96` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:34.825157 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702022a3-17f9-463d-b192-98c260cd9c81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ab9a532d467403aa288e3040406dd96)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:34.827060 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702022a3-17f9-463d-b192-98c260cd9c81 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:34.928807 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-702022a3-17f9-463d-b192-98c260cd9c81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ab9a532d467403aa288e3040406dd96', 'request_id': 'req-702022a3-17f9-463d-b192-98c260cd9c81'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:34.994509 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-702022a3-17f9-463d-b192-98c260cd9c81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ab9a532d467403aa288e3040406dd96', 'request_id': 'req-702022a3-17f9-463d-b192-98c260cd9c81'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:35.000676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-702022a3-17f9-463d-b192-98c260cd9c81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ab9a532d467403aa288e3040406dd96', 'request_id': 'req-702022a3-17f9-463d-b192-98c260cd9c81'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:35.011745 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-702022a3-17f9-463d-b192-98c260cd9c81 admin admin] The token cache is being invalidate because project 2ab9a532d467403aa288e3040406dd96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:06:35.012064 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-702022a3-17f9-463d-b192-98c260cd9c81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:35.012755 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-702022a3-17f9-463d-b192-98c260cd9c81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:35.014275 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1430/2838] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:06:34 2022] DELETE /identity/v3/projects/2ab9a532d467403aa288e3040406dd96 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:06:35.019247 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf7c53-7159-4941-a838-b39ae5a12860 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:35.019764 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf7c53-7159-4941-a838-b39ae5a12860 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:35.019882 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf7c53-7159-4941-a838-b39ae5a12860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:35.039698 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-bbcf7c53-7159-4941-a838-b39ae5a12860 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:06:35.040941 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-bbcf7c53-7159-4941-a838-b39ae5a12860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:35.057784 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:06:35.099804 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1409/2839] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:06:35.108136 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ec397-ffd6-490a-bae0-513df6ee59d3 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:35.108424 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ec397-ffd6-490a-bae0-513df6ee59d3 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:35.108724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ec397-ffd6-490a-bae0-513df6ee59d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:35.128741 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-0e8ec397-ffd6-490a-bae0-513df6ee59d3 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:06:35.136528 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-0e8ec397-ffd6-490a-bae0-513df6ee59d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:35.138353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1431/2840] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:06:35.142496 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf7c53-7159-4941-a838-b39ae5a12860 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:35.151877 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf7c53-7159-4941-a838-b39ae5a12860 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:35.156536 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de46ad41-fa54-4752-90b2-6d34ce5c7b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:35.157135 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46ad41-fa54-4752-90b2-6d34ce5c7b8f admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:35.157415 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46ad41-fa54-4752-90b2-6d34ce5c7b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:35.157689 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46ad41-fa54-4752-90b2-6d34ce5c7b8f admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:35.158333 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de46ad41-fa54-4752-90b2-6d34ce5c7b8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 01:06:35.158691 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de46ad41-fa54-4752-90b2-6d34ce5c7b8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:35.160867 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de46ad41-fa54-4752-90b2-6d34ce5c7b8f admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:35.173340 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1410/2841] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 01:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 01:06:35.177534 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ec397-ffd6-490a-bae0-513df6ee59d3 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:35.187140 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ec397-ffd6-490a-bae0-513df6ee59d3 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:35.191738 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44110eeb-2cb9-472d-8e50-1753110437b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:35.192334 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44110eeb-2cb9-472d-8e50-1753110437b1 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:35.192646 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44110eeb-2cb9-472d-8e50-1753110437b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:35.192920 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44110eeb-2cb9-472d-8e50-1753110437b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:35.193552 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44110eeb-2cb9-472d-8e50-1753110437b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:35.195452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44110eeb-2cb9-472d-8e50-1753110437b1 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:35.280603 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1432/2842] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:06:35 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:06:35.284836 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de46ad41-fa54-4752-90b2-6d34ce5c7b8f admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:35.296064 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de46ad41-fa54-4752-90b2-6d34ce5c7b8f admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:35.353284 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1b9c4-1642-4b24-a26d-445ccd8263c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:35.353903 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1b9c4-1642-4b24-a26d-445ccd8263c7 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:35.357020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1b9c4-1642-4b24-a26d-445ccd8263c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:35.357020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1b9c4-1642-4b24-a26d-445ccd8263c7 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:35.357020 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1b9c4-1642-4b24-a26d-445ccd8263c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:35.357576 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1b9c4-1642-4b24-a26d-445ccd8263c7 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:35.446363 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1411/2843] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:06:35 2022] POST /identity/v3/users => generated 287 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:06:35.450792 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44110eeb-2cb9-472d-8e50-1753110437b1 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:35.461280 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44110eeb-2cb9-472d-8e50-1753110437b1 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:35.466563 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fb6a7-c066-4c3d-a360-a487861b637d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:35.467167 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fb6a7-c066-4c3d-a360-a487861b637d admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:35.467452 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fb6a7-c066-4c3d-a360-a487861b637d admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:35.467729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fb6a7-c066-4c3d-a360-a487861b637d admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:35.468379 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0fb6a7-c066-4c3d-a360-a487861b637d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:35.470021 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0fb6a7-c066-4c3d-a360-a487861b637d admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:35.479889 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1433/2844] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:06:35.483838 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1b9c4-1642-4b24-a26d-445ccd8263c7 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:35.494198 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1b9c4-1642-4b24-a26d-445ccd8263c7 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:35.498939 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59088a09-514e-4195-ba7d-7edcc1139721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:35.499544 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59088a09-514e-4195-ba7d-7edcc1139721 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:35.500455 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59088a09-514e-4195-ba7d-7edcc1139721 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:35.500764 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59088a09-514e-4195-ba7d-7edcc1139721 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:35.501410 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59088a09-514e-4195-ba7d-7edcc1139721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:35.503066 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59088a09-514e-4195-ba7d-7edcc1139721 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:35.514249 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1412/2845] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:06:35.518433 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fb6a7-c066-4c3d-a360-a487861b637d admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:35.528956 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fb6a7-c066-4c3d-a360-a487861b637d admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:35.533628 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de86dd-fa1f-4b91-9ae6-bdd223e8bb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:35.534302 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de86dd-fa1f-4b91-9ae6-bdd223e8bb0a admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:35.534585 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de86dd-fa1f-4b91-9ae6-bdd223e8bb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:35.534861 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de86dd-fa1f-4b91-9ae6-bdd223e8bb0a admin admin] PATH_INFO: `/v3/projects/5244e31a6b4f4102834cd0a79c7c780f/users/8e1bcdffe9d94b679aea7106dccf87b9/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:35.550132 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de86dd-fa1f-4b91-9ae6-bdd223e8bb0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5244e31a6b4f4102834cd0a79c7c780f, user_id=8e1bcdffe9d94b679aea7106dccf87b9, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:35.553111 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de86dd-fa1f-4b91-9ae6-bdd223e8bb0a admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:35.621658 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1434/2846] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:06:35 2022] PUT /identity/v3/projects/5244e31a6b4f4102834cd0a79c7c780f/users/8e1bcdffe9d94b679aea7106dccf87b9/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 01:06:35.632495 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8692b81-51f2-4017-b9d9-417a16adc652 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:35.632799 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8692b81-51f2-4017-b9d9-417a16adc652 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:35.633190 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8692b81-51f2-4017-b9d9-417a16adc652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:35.661765 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-e8692b81-51f2-4017-b9d9-417a16adc652 None None] MFA Rules not processed for user `8e1bcdffe9d94b679aea7106dccf87b9`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:06:35.663290 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-e8692b81-51f2-4017-b9d9-417a16adc652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:35.729682 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1413/2847] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:06:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:06:35.741045 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de86dd-fa1f-4b91-9ae6-bdd223e8bb0a admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:35.742749 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-86de86dd-fa1f-4b91-9ae6-bdd223e8bb0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:35.765844 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:06:35.805766 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de86dd-fa1f-4b91-9ae6-bdd223e8bb0a admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:35.810518 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1cd0e-0e33-45a9-9f19-a1b4457af4a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:35.811126 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1cd0e-0e33-45a9-9f19-a1b4457af4a0 service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:35.811526 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1cd0e-0e33-45a9-9f19-a1b4457af4a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:35.811806 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1cd0e-0e33-45a9-9f19-a1b4457af4a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:35.822767 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc1cd0e-0e33-45a9-9f19-a1b4457af4a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:35.824376 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc1cd0e-0e33-45a9-9f19-a1b4457af4a0 service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:35.835556 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1435/2848] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:06:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:06:35.862057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8692b81-51f2-4017-b9d9-417a16adc652 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:35.882583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:06:35.927196 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8692b81-51f2-4017-b9d9-417a16adc652 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:35.931819 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae952b7e-508a-48e4-bacf-8081e06021c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:35.932454 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae952b7e-508a-48e4-bacf-8081e06021c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:35.932760 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae952b7e-508a-48e4-bacf-8081e06021c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:35.933036 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae952b7e-508a-48e4-bacf-8081e06021c6 admin admin] PATH_INFO: `/v3/users/8e1bcdffe9d94b679aea7106dccf87b9` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:35.934193 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae952b7e-508a-48e4-bacf-8081e06021c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e1bcdffe9d94b679aea7106dccf87b9)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:35.936161 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae952b7e-508a-48e4-bacf-8081e06021c6 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:36.167845 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-ae952b7e-508a-48e4-bacf-8081e06021c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e1bcdffe9d94b679aea7106dccf87b9'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:36.181521 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-ae952b7e-508a-48e4-bacf-8081e06021c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e1bcdffe9d94b679aea7106dccf87b9'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:36.250206 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-ae952b7e-508a-48e4-bacf-8081e06021c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e1bcdffe9d94b679aea7106dccf87b9'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:36.250830 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-ae952b7e-508a-48e4-bacf-8081e06021c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e1bcdffe9d94b679aea7106dccf87b9'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:36.251445 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-ae952b7e-508a-48e4-bacf-8081e06021c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e1bcdffe9d94b679aea7106dccf87b9'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:36.264961 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1414/2849] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:06:35 2022] DELETE /identity/v3/users/8e1bcdffe9d94b679aea7106dccf87b9 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:06:36.269555 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3bcb7-a22e-4dc4-9a43-7b5ae19d8d27 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:36.270037 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3bcb7-a22e-4dc4-9a43-7b5ae19d8d27 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:36.270174 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3bcb7-a22e-4dc4-9a43-7b5ae19d8d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:36.289729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-17b3bcb7-a22e-4dc4-9a43-7b5ae19d8d27 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:06:36.290979 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-17b3bcb7-a22e-4dc4-9a43-7b5ae19d8d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:36.352284 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1436/2850] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:06:36.363162 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae952b7e-508a-48e4-bacf-8081e06021c6 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:36.364999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-ae952b7e-508a-48e4-bacf-8081e06021c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:36.429840 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae952b7e-508a-48e4-bacf-8081e06021c6 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:36.439820 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c0cf0-88d6-4a7c-bc62-247878377e2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:36.440450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c0cf0-88d6-4a7c-bc62-247878377e2a service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:36.440767 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c0cf0-88d6-4a7c-bc62-247878377e2a service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:36.441037 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c0cf0-88d6-4a7c-bc62-247878377e2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:36.450986 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c0cf0-88d6-4a7c-bc62-247878377e2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:36.452440 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c0cf0-88d6-4a7c-bc62-247878377e2a service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:36.466844 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1415/2851] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:06:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:06:36.927436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3bcb7-a22e-4dc4-9a43-7b5ae19d8d27 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:36.930212 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-17b3bcb7-a22e-4dc4-9a43-7b5ae19d8d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:36.953702 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3bcb7-a22e-4dc4-9a43-7b5ae19d8d27 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:36.959203 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b581c88-1447-48df-86d0-b28e3e9be9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:36.959872 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b581c88-1447-48df-86d0-b28e3e9be9b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:36.960155 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b581c88-1447-48df-86d0-b28e3e9be9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:36.960428 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b581c88-1447-48df-86d0-b28e3e9be9b9 admin admin] PATH_INFO: `/v3/projects/5244e31a6b4f4102834cd0a79c7c780f` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:36.961903 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b581c88-1447-48df-86d0-b28e3e9be9b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5244e31a6b4f4102834cd0a79c7c780f)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:36.963612 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b581c88-1447-48df-86d0-b28e3e9be9b9 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:37.086334 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-8b581c88-1447-48df-86d0-b28e3e9be9b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5244e31a6b4f4102834cd0a79c7c780f', 'request_id': 'req-8b581c88-1447-48df-86d0-b28e3e9be9b9'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:37.171559 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-8b581c88-1447-48df-86d0-b28e3e9be9b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5244e31a6b4f4102834cd0a79c7c780f', 'request_id': 'req-8b581c88-1447-48df-86d0-b28e3e9be9b9'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:37.177860 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-8b581c88-1447-48df-86d0-b28e3e9be9b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5244e31a6b4f4102834cd0a79c7c780f', 'request_id': 'req-8b581c88-1447-48df-86d0-b28e3e9be9b9'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:37.189100 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-8b581c88-1447-48df-86d0-b28e3e9be9b9 admin admin] The token cache is being invalidate because project 5244e31a6b4f4102834cd0a79c7c780f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:06:37.189394 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-8b581c88-1447-48df-86d0-b28e3e9be9b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:37.190053 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-8b581c88-1447-48df-86d0-b28e3e9be9b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:37.191604 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1437/2852] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:06:36 2022] DELETE /identity/v3/projects/5244e31a6b4f4102834cd0a79c7c780f => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:06:37.196560 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee97d8-853a-47e8-96d2-59e56174715c None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:37.197047 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee97d8-853a-47e8-96d2-59e56174715c None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:37.197148 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee97d8-853a-47e8-96d2-59e56174715c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:37.218043 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-36ee97d8-853a-47e8-96d2-59e56174715c None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:06:37.219256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-36ee97d8-853a-47e8-96d2-59e56174715c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:37.252274 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:06:37.277014 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1416/2853] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:06:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:06:37.285471 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21690a92-bc21-4488-acaa-ded8c84a3bfa None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:37.286002 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21690a92-bc21-4488-acaa-ded8c84a3bfa None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:37.286308 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21690a92-bc21-4488-acaa-ded8c84a3bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:37.307391 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-21690a92-bc21-4488-acaa-ded8c84a3bfa None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:06:37.308640 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-21690a92-bc21-4488-acaa-ded8c84a3bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:37.316413 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1438/2854] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:06:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:06:37.322736 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee97d8-853a-47e8-96d2-59e56174715c None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:37.334246 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee97d8-853a-47e8-96d2-59e56174715c None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:37.339109 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d977c9b-cd20-4a86-b3e8-c15f64a689a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:37.339953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d977c9b-cd20-4a86-b3e8-c15f64a689a3 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:37.340336 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d977c9b-cd20-4a86-b3e8-c15f64a689a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:37.341246 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d977c9b-cd20-4a86-b3e8-c15f64a689a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:37.342008 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d977c9b-cd20-4a86-b3e8-c15f64a689a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88253) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 01:06:37.342411 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d977c9b-cd20-4a86-b3e8-c15f64a689a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:37.344194 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d977c9b-cd20-4a86-b3e8-c15f64a689a3 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:37.357382 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1417/2855] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 01:06:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 01:06:37.361888 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21690a92-bc21-4488-acaa-ded8c84a3bfa None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:37.371484 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21690a92-bc21-4488-acaa-ded8c84a3bfa None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:37.376269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5813d-c545-4189-bb77-3217ef0cfcdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:37.376919 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5813d-c545-4189-bb77-3217ef0cfcdf admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:37.377249 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5813d-c545-4189-bb77-3217ef0cfcdf admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:37.377529 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5813d-c545-4189-bb77-3217ef0cfcdf admin admin] PATH_INFO: `/v3/projects` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:37.378168 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5813d-c545-4189-bb77-3217ef0cfcdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:37.380111 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5813d-c545-4189-bb77-3217ef0cfcdf admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:37.562220 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1439/2856] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:06:37 2022] POST /identity/v3/projects => generated 381 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 01:06:37.566638 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d977c9b-cd20-4a86-b3e8-c15f64a689a3 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:37.577458 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d977c9b-cd20-4a86-b3e8-c15f64a689a3 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:37.634386 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ecba56-b00c-4cbc-af6d-2c7d0ee74063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:37.634999 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ecba56-b00c-4cbc-af6d-2c7d0ee74063 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:37.635282 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ecba56-b00c-4cbc-af6d-2c7d0ee74063 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:37.635560 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ecba56-b00c-4cbc-af6d-2c7d0ee74063 admin admin] PATH_INFO: `/v3/users` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:37.636195 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ecba56-b00c-4cbc-af6d-2c7d0ee74063 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:37.638805 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ecba56-b00c-4cbc-af6d-2c7d0ee74063 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:37.777988 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1418/2857] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:06:37 2022] POST /identity/v3/users => generated 291 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:06:37.782642 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5813d-c545-4189-bb77-3217ef0cfcdf admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:37.793222 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5813d-c545-4189-bb77-3217ef0cfcdf admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:37.797737 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131005e-5a81-4539-9a48-c4195fac11ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:37.798340 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131005e-5a81-4539-9a48-c4195fac11ed admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:37.798623 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131005e-5a81-4539-9a48-c4195fac11ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:37.798893 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131005e-5a81-4539-9a48-c4195fac11ed admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:37.799538 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9131005e-5a81-4539-9a48-c4195fac11ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:37.801188 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9131005e-5a81-4539-9a48-c4195fac11ed admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:37.811787 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1440/2858] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:06:37.815656 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ecba56-b00c-4cbc-af6d-2c7d0ee74063 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:37.826522 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ecba56-b00c-4cbc-af6d-2c7d0ee74063 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:37.831530 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95451e8a-1b69-4200-bd4f-37106454cc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:37.832141 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95451e8a-1b69-4200-bd4f-37106454cc91 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:37.832423 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95451e8a-1b69-4200-bd4f-37106454cc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:37.832738 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95451e8a-1b69-4200-bd4f-37106454cc91 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:37.833414 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95451e8a-1b69-4200-bd4f-37106454cc91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:37.835096 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95451e8a-1b69-4200-bd4f-37106454cc91 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:37.845864 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1419/2859] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:06:37.849988 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131005e-5a81-4539-9a48-c4195fac11ed admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:37.860205 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131005e-5a81-4539-9a48-c4195fac11ed admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:37.864900 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf755bdf-01b7-410f-b3a3-f50d7b2d248d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:37.874010 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf755bdf-01b7-410f-b3a3-f50d7b2d248d admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:37.874010 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf755bdf-01b7-410f-b3a3-f50d7b2d248d admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:37.874010 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf755bdf-01b7-410f-b3a3-f50d7b2d248d admin admin] PATH_INFO: `/v3/projects/0f048337938642b2bf9165db865e8edb/users/a4b35218c0034a58bf245444b877ff8e/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:37.874010 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:06:37.881605 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf755bdf-01b7-410f-b3a3-f50d7b2d248d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f048337938642b2bf9165db865e8edb, user_id=a4b35218c0034a58bf245444b877ff8e, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:37.884276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf755bdf-01b7-410f-b3a3-f50d7b2d248d admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:38.064225 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1441/2860] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:06:37 2022] PUT /identity/v3/projects/0f048337938642b2bf9165db865e8edb/users/a4b35218c0034a58bf245444b877ff8e/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:06:38.064768 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59c429-7f7e-484f-b5a3-900d8fcc5461 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:38.064768 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59c429-7f7e-484f-b5a3-900d8fcc5461 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:38.064768 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59c429-7f7e-484f-b5a3-900d8fcc5461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:38.072498 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-7e59c429-7f7e-484f-b5a3-900d8fcc5461 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:06:38.073639 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-7e59c429-7f7e-484f-b5a3-900d8fcc5461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:38.131121 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1420/2861] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:06:38.156402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf755bdf-01b7-410f-b3a3-f50d7b2d248d admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:38.159770 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-cf755bdf-01b7-410f-b3a3-f50d7b2d248d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:38.218392 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf755bdf-01b7-410f-b3a3-f50d7b2d248d admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:38.222014 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58100cd2-06fd-45c4-b699-10fcb6b23dd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:38.222636 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58100cd2-06fd-45c4-b699-10fcb6b23dd8 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:38.222917 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58100cd2-06fd-45c4-b699-10fcb6b23dd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:38.223222 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58100cd2-06fd-45c4-b699-10fcb6b23dd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:38.233269 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58100cd2-06fd-45c4-b699-10fcb6b23dd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:38.245247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58100cd2-06fd-45c4-b699-10fcb6b23dd8 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:38.245247 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1442/2862] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:06:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:06:40.375226 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572e9e2-3a89-4ad1-b7c8-c39179ca0b45 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:40.375770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572e9e2-3a89-4ad1-b7c8-c39179ca0b45 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:40.375770 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572e9e2-3a89-4ad1-b7c8-c39179ca0b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:40.404055 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-1572e9e2-3a89-4ad1-b7c8-c39179ca0b45 None None] MFA Rules not processed for user `a4b35218c0034a58bf245444b877ff8e`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:06:40.405298 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-1572e9e2-3a89-4ad1-b7c8-c39179ca0b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:40.447233 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:06:40.478624 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1421/2863] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:06:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:06:40.493196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58100cd2-06fd-45c4-b699-10fcb6b23dd8 service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:40.493196 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-58100cd2-06fd-45c4-b699-10fcb6b23dd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:40.552203 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58100cd2-06fd-45c4-b699-10fcb6b23dd8 service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:40.556239 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87fc574-9a43-4847-973a-331990de9d65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c88202614775418ba94f478f03431a9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:40.556983 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87fc574-9a43-4847-973a-331990de9d65 service cinder] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:40.557254 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87fc574-9a43-4847-973a-331990de9d65 service cinder] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:40.557861 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87fc574-9a43-4847-973a-331990de9d65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:40.567358 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87fc574-9a43-4847-973a-331990de9d65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:40.568834 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87fc574-9a43-4847-973a-331990de9d65 service cinder] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:40.577830 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1443/2864] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:06:40 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:06:42.686874 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059735d3-385f-4e2e-9c65-b07f9f923502 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:42.687400 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059735d3-385f-4e2e-9c65-b07f9f923502 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:42.687400 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059735d3-385f-4e2e-9c65-b07f9f923502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:42.706414 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-059735d3-385f-4e2e-9c65-b07f9f923502 None None] MFA Rules not processed for user `a4b35218c0034a58bf245444b877ff8e`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:06:42.707528 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-059735d3-385f-4e2e-9c65-b07f9f923502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:42.717489 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1422/2865] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:06:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 01:06:42.976811 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87fc574-9a43-4847-973a-331990de9d65 service cinder] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:42.995339 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87fc574-9a43-4847-973a-331990de9d65 service cinder] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:43.003402 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52210d3b-bef2-4e90-a0d1-e48dc0e563d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:43.004492 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52210d3b-bef2-4e90-a0d1-e48dc0e563d5 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:43.005732 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52210d3b-bef2-4e90-a0d1-e48dc0e563d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:43.006344 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52210d3b-bef2-4e90-a0d1-e48dc0e563d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:43.008633 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-52210d3b-bef2-4e90-a0d1-e48dc0e563d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:43.025194 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52210d3b-bef2-4e90-a0d1-e48dc0e563d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:43.027017 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52210d3b-bef2-4e90-a0d1-e48dc0e563d5 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:43.040283 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1444/2866] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:06:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:06:43.574007 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059735d3-385f-4e2e-9c65-b07f9f923502 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:43.575649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-059735d3-385f-4e2e-9c65-b07f9f923502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:43.649670 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059735d3-385f-4e2e-9c65-b07f9f923502 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:43.654534 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da643a35-0943-40dc-8d2a-9cc05d566774 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2876f4182f0949039b862f053533fe85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:43.655139 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da643a35-0943-40dc-8d2a-9cc05d566774 service placement] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:43.655422 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da643a35-0943-40dc-8d2a-9cc05d566774 service placement] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:43.655718 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da643a35-0943-40dc-8d2a-9cc05d566774 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:43.657432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-da643a35-0943-40dc-8d2a-9cc05d566774 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:43.669527 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da643a35-0943-40dc-8d2a-9cc05d566774 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:43.671115 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da643a35-0943-40dc-8d2a-9cc05d566774 service placement] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:43.680863 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1423/2867] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:06:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 01:06:47.913456 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52210d3b-bef2-4e90-a0d1-e48dc0e563d5 service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:47.915674 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-52210d3b-bef2-4e90-a0d1-e48dc0e563d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:47.931796 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52210d3b-bef2-4e90-a0d1-e48dc0e563d5 service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:47.937314 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766b0f2-de16-4a0c-8338-af8d254a3956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:47.937959 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766b0f2-de16-4a0c-8338-af8d254a3956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:47.938301 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766b0f2-de16-4a0c-8338-af8d254a3956 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:47.938628 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766b0f2-de16-4a0c-8338-af8d254a3956 admin admin] PATH_INFO: `/v3/users/83076e7b11fb446ca50f23fe8e6e7ab6` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:47.939812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7766b0f2-de16-4a0c-8338-af8d254a3956 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83076e7b11fb446ca50f23fe8e6e7ab6)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:47.941897 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7766b0f2-de16-4a0c-8338-af8d254a3956 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:48.140600 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-7766b0f2-de16-4a0c-8338-af8d254a3956 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83076e7b11fb446ca50f23fe8e6e7ab6'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:48.141649 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-7766b0f2-de16-4a0c-8338-af8d254a3956 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83076e7b11fb446ca50f23fe8e6e7ab6'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:48.155611 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-7766b0f2-de16-4a0c-8338-af8d254a3956 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83076e7b11fb446ca50f23fe8e6e7ab6'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:48.159705 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:06:48.212125 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-7766b0f2-de16-4a0c-8338-af8d254a3956 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83076e7b11fb446ca50f23fe8e6e7ab6'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:48.213348 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-7766b0f2-de16-4a0c-8338-af8d254a3956 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83076e7b11fb446ca50f23fe8e6e7ab6'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:48.229729 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1445/2868] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:06:47 2022] DELETE /identity/v3/users/83076e7b11fb446ca50f23fe8e6e7ab6 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:06:48.522057 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da643a35-0943-40dc-8d2a-9cc05d566774 service placement] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:48.543030 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da643a35-0943-40dc-8d2a-9cc05d566774 service placement] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:48.604389 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b97be0-de84-4f2d-871e-c6dde07da8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:48.605208 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b97be0-de84-4f2d-871e-c6dde07da8af admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:48.605554 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b97be0-de84-4f2d-871e-c6dde07da8af admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:48.606351 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b97be0-de84-4f2d-871e-c6dde07da8af admin admin] PATH_INFO: `/v3/projects/a223be1c2f6a4cd281b20b0bc7281554` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:48.607600 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b97be0-de84-4f2d-871e-c6dde07da8af admin admin] RBAC: Authorizing `identity:delete_project(project_id=a223be1c2f6a4cd281b20b0bc7281554)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:48.609636 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b97be0-de84-4f2d-871e-c6dde07da8af admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:48.743080 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-36b97be0-de84-4f2d-871e-c6dde07da8af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a223be1c2f6a4cd281b20b0bc7281554', 'request_id': 'req-36b97be0-de84-4f2d-871e-c6dde07da8af'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:48.823125 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-36b97be0-de84-4f2d-871e-c6dde07da8af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a223be1c2f6a4cd281b20b0bc7281554', 'request_id': 'req-36b97be0-de84-4f2d-871e-c6dde07da8af'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:48.830422 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-36b97be0-de84-4f2d-871e-c6dde07da8af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a223be1c2f6a4cd281b20b0bc7281554', 'request_id': 'req-36b97be0-de84-4f2d-871e-c6dde07da8af'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:48.831674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:06:48.843943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-36b97be0-de84-4f2d-871e-c6dde07da8af admin admin] The token cache is being invalidate because project a223be1c2f6a4cd281b20b0bc7281554 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:06:48.844236 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-36b97be0-de84-4f2d-871e-c6dde07da8af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:48.844899 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-36b97be0-de84-4f2d-871e-c6dde07da8af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:48.846400 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1424/2869] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:06:48 2022] DELETE /identity/v3/projects/a223be1c2f6a4cd281b20b0bc7281554 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:06:48.850573 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766b0f2-de16-4a0c-8338-af8d254a3956 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:48.852040 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-7766b0f2-de16-4a0c-8338-af8d254a3956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:48.865323 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766b0f2-de16-4a0c-8338-af8d254a3956 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:48.925549 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb687f-6199-4002-a75c-e0684b915bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:48.926300 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb687f-6199-4002-a75c-e0684b915bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:48.926642 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb687f-6199-4002-a75c-e0684b915bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:48.926993 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb687f-6199-4002-a75c-e0684b915bd4 admin admin] PATH_INFO: `/v3/users/d770a26f90c740a2bbfc2d4dee853da2` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:48.928573 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb687f-6199-4002-a75c-e0684b915bd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d770a26f90c740a2bbfc2d4dee853da2)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:48.932088 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb687f-6199-4002-a75c-e0684b915bd4 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:49.156701 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-1cdb687f-6199-4002-a75c-e0684b915bd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd770a26f90c740a2bbfc2d4dee853da2'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:49.157352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-1cdb687f-6199-4002-a75c-e0684b915bd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd770a26f90c740a2bbfc2d4dee853da2'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:49.169494 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-1cdb687f-6199-4002-a75c-e0684b915bd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd770a26f90c740a2bbfc2d4dee853da2'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:49.250990 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-1cdb687f-6199-4002-a75c-e0684b915bd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd770a26f90c740a2bbfc2d4dee853da2'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:49.251627 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-1cdb687f-6199-4002-a75c-e0684b915bd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd770a26f90c740a2bbfc2d4dee853da2'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:49.264966 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1446/2870] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:06:48 2022] DELETE /identity/v3/users/d770a26f90c740a2bbfc2d4dee853da2 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:06:49.554146 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b97be0-de84-4f2d-871e-c6dde07da8af admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:06:49.555674 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-36b97be0-de84-4f2d-871e-c6dde07da8af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:06:49.568390 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b97be0-de84-4f2d-871e-c6dde07da8af admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:06:49.627408 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563a925-bc70-435c-82b4-dbdb2dfb694d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:06:49.627969 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563a925-bc70-435c-82b4-dbdb2dfb694d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:06:49.628253 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563a925-bc70-435c-82b4-dbdb2dfb694d admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:06:49.628564 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563a925-bc70-435c-82b4-dbdb2dfb694d admin admin] PATH_INFO: `/v3/projects/5d843f2ba1a24c9ca97a215c8cbe2240` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:06:49.629705 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563a925-bc70-435c-82b4-dbdb2dfb694d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d843f2ba1a24c9ca97a215c8cbe2240)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:06:49.631637 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563a925-bc70-435c-82b4-dbdb2dfb694d admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:06:49.772051 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0563a925-bc70-435c-82b4-dbdb2dfb694d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d843f2ba1a24c9ca97a215c8cbe2240', 'request_id': 'req-0563a925-bc70-435c-82b4-dbdb2dfb694d'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:49.891345 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0563a925-bc70-435c-82b4-dbdb2dfb694d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d843f2ba1a24c9ca97a215c8cbe2240', 'request_id': 'req-0563a925-bc70-435c-82b4-dbdb2dfb694d'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:49.897932 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0563a925-bc70-435c-82b4-dbdb2dfb694d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d843f2ba1a24c9ca97a215c8cbe2240', 'request_id': 'req-0563a925-bc70-435c-82b4-dbdb2dfb694d'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:49.909203 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0563a925-bc70-435c-82b4-dbdb2dfb694d admin admin] The token cache is being invalidate because project 5d843f2ba1a24c9ca97a215c8cbe2240 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:06:49.909482 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0563a925-bc70-435c-82b4-dbdb2dfb694d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:49.910109 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-0563a925-bc70-435c-82b4-dbdb2dfb694d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:06:49.911572 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1425/2871] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:06:49 2022] DELETE /identity/v3/projects/5d843f2ba1a24c9ca97a215c8cbe2240 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:06:59.260645 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=59,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:06:59.906083 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:DELETE=9,keystone:SELECT=54,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:07:01.534267 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb687f-6199-4002-a75c-e0684b915bd4 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:07:01.535682 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-1cdb687f-6199-4002-a75c-e0684b915bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:07:01.607994 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb687f-6199-4002-a75c-e0684b915bd4 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:07:01.668230 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59fdbfc-70d2-4607-bc55-553147549d59 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2876f4182f0949039b862f053533fe85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:07:01.668870 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fdbfc-70d2-4607-bc55-553147549d59 service placement] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:07:01.669151 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fdbfc-70d2-4607-bc55-553147549d59 service placement] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:07:01.669423 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fdbfc-70d2-4607-bc55-553147549d59 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:07:01.671276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f59fdbfc-70d2-4607-bc55-553147549d59 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:07:01.683813 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59fdbfc-70d2-4607-bc55-553147549d59 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:07:01.685466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59fdbfc-70d2-4607-bc55-553147549d59 service placement] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:07:01.695978 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1447/2872] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:07:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:07:11.691521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=58 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:07:53.703426 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e92aa9-f440-42ff-be71-cadb4a77db62 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:07:53.703968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e92aa9-f440-42ff-be71-cadb4a77db62 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:07:53.703968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e92aa9-f440-42ff-be71-cadb4a77db62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:07:53.722248 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-d5e92aa9-f440-42ff-be71-cadb4a77db62 None None] MFA Rules not processed for user `a4b35218c0034a58bf245444b877ff8e`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:07:53.723924 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-d5e92aa9-f440-42ff-be71-cadb4a77db62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:07:53.776697 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1426/2873] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:07:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:08:03.770096 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=25 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:09:02.615086 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59fdbfc-70d2-4607-bc55-553147549d59 service placement] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:09:02.616810 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f59fdbfc-70d2-4607-bc55-553147549d59 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:09:02.683087 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59fdbfc-70d2-4607-bc55-553147549d59 service placement] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:09:02.687521 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6798de1-4c15-4a74-b2ba-a566fa7ed184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:09:02.688154 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6798de1-4c15-4a74-b2ba-a566fa7ed184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:02.688436 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6798de1-4c15-4a74-b2ba-a566fa7ed184 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:02.688743 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6798de1-4c15-4a74-b2ba-a566fa7ed184 admin admin] PATH_INFO: `/v3/users/a4b35218c0034a58bf245444b877ff8e` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:02.689862 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6798de1-4c15-4a74-b2ba-a566fa7ed184 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4b35218c0034a58bf245444b877ff8e)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:09:02.691819 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6798de1-4c15-4a74-b2ba-a566fa7ed184 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:09:03.015606 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-e6798de1-4c15-4a74-b2ba-a566fa7ed184 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4b35218c0034a58bf245444b877ff8e'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:09:03.016186 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-e6798de1-4c15-4a74-b2ba-a566fa7ed184 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4b35218c0034a58bf245444b877ff8e'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:09:03.028857 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-e6798de1-4c15-4a74-b2ba-a566fa7ed184 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4b35218c0034a58bf245444b877ff8e'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:09:03.147192 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-e6798de1-4c15-4a74-b2ba-a566fa7ed184 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4b35218c0034a58bf245444b877ff8e'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:09:03.159877 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-e6798de1-4c15-4a74-b2ba-a566fa7ed184 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4b35218c0034a58bf245444b877ff8e'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:09:03.162104 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1448/2874] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:09:02 2022] DELETE /identity/v3/users/a4b35218c0034a58bf245444b877ff8e => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:09:03.444075 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e92aa9-f440-42ff-be71-cadb4a77db62 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:09:03.459160 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e92aa9-f440-42ff-be71-cadb4a77db62 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:09:03.521369 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0199c-f363-4386-a04f-36bbcd8ae020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:09:03.521906 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0199c-f363-4386-a04f-36bbcd8ae020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:03.522184 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0199c-f363-4386-a04f-36bbcd8ae020 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:03.522455 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0199c-f363-4386-a04f-36bbcd8ae020 admin admin] PATH_INFO: `/v3/projects/0f048337938642b2bf9165db865e8edb` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:03.523568 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca0199c-f363-4386-a04f-36bbcd8ae020 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f048337938642b2bf9165db865e8edb)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:09:03.525500 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca0199c-f363-4386-a04f-36bbcd8ae020 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:09:03.622038 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-3ca0199c-f363-4386-a04f-36bbcd8ae020 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f048337938642b2bf9165db865e8edb', 'request_id': 'req-3ca0199c-f363-4386-a04f-36bbcd8ae020'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:09:03.700773 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-3ca0199c-f363-4386-a04f-36bbcd8ae020 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f048337938642b2bf9165db865e8edb', 'request_id': 'req-3ca0199c-f363-4386-a04f-36bbcd8ae020'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:09:03.707366 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-3ca0199c-f363-4386-a04f-36bbcd8ae020 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f048337938642b2bf9165db865e8edb', 'request_id': 'req-3ca0199c-f363-4386-a04f-36bbcd8ae020'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:09:03.717574 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-3ca0199c-f363-4386-a04f-36bbcd8ae020 admin admin] The token cache is being invalidate because project 0f048337938642b2bf9165db865e8edb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88253) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:09:03.717856 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-3ca0199c-f363-4386-a04f-36bbcd8ae020 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:09:03.718479 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-3ca0199c-f363-4386-a04f-36bbcd8ae020 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:09:03.719965 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1427/2875] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:09:03 2022] DELETE /identity/v3/projects/0f048337938642b2bf9165db865e8edb => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:09:03.725157 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4ab71-4aa1-4e2f-93ee-1af2c16405f5 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:03.725666 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4ab71-4aa1-4e2f-93ee-1af2c16405f5 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:03.725783 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4ab71-4aa1-4e2f-93ee-1af2c16405f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:03.745686 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-5eb4ab71-4aa1-4e2f-93ee-1af2c16405f5 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:09:03.746916 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-5eb4ab71-4aa1-4e2f-93ee-1af2c16405f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:09:03.805404 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1449/2876] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:09:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:09:03.813228 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60b182-b53d-4893-b5b7-1736794fb3eb None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:03.814184 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60b182-b53d-4893-b5b7-1736794fb3eb None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:03.814184 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60b182-b53d-4893-b5b7-1736794fb3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:03.834472 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-ab60b182-b53d-4893-b5b7-1736794fb3eb None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:09:03.835705 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-ab60b182-b53d-4893-b5b7-1736794fb3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:09:03.843757 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1428/2877] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:09:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:09:03.848155 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4ab71-4aa1-4e2f-93ee-1af2c16405f5 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:09:03.858683 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4ab71-4aa1-4e2f-93ee-1af2c16405f5 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:09:03.863307 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218e357-51a8-476b-9868-a6e042e721ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:09:03.863913 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218e357-51a8-476b-9868-a6e042e721ee admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:03.864193 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218e357-51a8-476b-9868-a6e042e721ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:03.864565 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218e357-51a8-476b-9868-a6e042e721ee admin admin] PATH_INFO: `/v3/domains` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:03.865178 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f218e357-51a8-476b-9868-a6e042e721ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88254) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 01:09:03.865483 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f218e357-51a8-476b-9868-a6e042e721ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:09:03.867077 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f218e357-51a8-476b-9868-a6e042e721ee admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:09:03.878672 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1450/2878] 10.222.0.59 () {58 vars in 1153 bytes} [Fri Aug 26 01:09:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 01:09:03.883252 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60b182-b53d-4893-b5b7-1736794fb3eb None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:09:03.892489 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60b182-b53d-4893-b5b7-1736794fb3eb None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:09:03.897797 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aaac71-be61-4ca6-80af-900c4745e6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:09:03.898394 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aaac71-be61-4ca6-80af-900c4745e6c4 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:03.898676 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aaac71-be61-4ca6-80af-900c4745e6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:03.898945 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aaac71-be61-4ca6-80af-900c4745e6c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:03.899687 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aaac71-be61-4ca6-80af-900c4745e6c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:09:03.901601 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aaac71-be61-4ca6-80af-900c4745e6c4 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:09:04.012208 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1429/2879] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:09:03 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:09:04.025494 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218e357-51a8-476b-9868-a6e042e721ee admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:09:04.036030 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218e357-51a8-476b-9868-a6e042e721ee admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:09:04.040903 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:09:04.094614 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a91c9-4b3f-4c6f-a0ef-978e9ea093f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:09:04.095228 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a91c9-4b3f-4c6f-a0ef-978e9ea093f5 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:04.095511 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a91c9-4b3f-4c6f-a0ef-978e9ea093f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:04.095788 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a91c9-4b3f-4c6f-a0ef-978e9ea093f5 admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:04.096416 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a91c9-4b3f-4c6f-a0ef-978e9ea093f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:09:04.098384 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a91c9-4b3f-4c6f-a0ef-978e9ea093f5 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:09:04.184347 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1451/2880] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:09:04 2022] POST /identity/v3/users => generated 292 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:09:04.188450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aaac71-be61-4ca6-80af-900c4745e6c4 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:09:04.199615 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aaac71-be61-4ca6-80af-900c4745e6c4 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:09:04.204320 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993db51-31bb-4f62-996e-b6a6e69f53fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:09:04.206338 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993db51-31bb-4f62-996e-b6a6e69f53fc admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:04.206338 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993db51-31bb-4f62-996e-b6a6e69f53fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:04.206338 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993db51-31bb-4f62-996e-b6a6e69f53fc admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:04.206338 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993db51-31bb-4f62-996e-b6a6e69f53fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:09:04.207695 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993db51-31bb-4f62-996e-b6a6e69f53fc admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:09:04.218424 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1430/2881] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:09:04.222206 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a91c9-4b3f-4c6f-a0ef-978e9ea093f5 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:09:04.232305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a91c9-4b3f-4c6f-a0ef-978e9ea093f5 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:09:04.236897 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e0217f-b234-4e9f-ba9b-7807bd894b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:09:04.237527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0217f-b234-4e9f-ba9b-7807bd894b07 admin admin] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:04.237808 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0217f-b234-4e9f-ba9b-7807bd894b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:04.238087 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0217f-b234-4e9f-ba9b-7807bd894b07 admin admin] PATH_INFO: `/v3/roles` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:04.238727 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e0217f-b234-4e9f-ba9b-7807bd894b07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:09:04.240321 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e0217f-b234-4e9f-ba9b-7807bd894b07 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:09:04.251475 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1452/2882] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:09:04.256263 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993db51-31bb-4f62-996e-b6a6e69f53fc admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:09:04.266330 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993db51-31bb-4f62-996e-b6a6e69f53fc admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:09:04.271430 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b687b-3604-4025-8cbc-05e07a27b62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:09:04.272073 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b687b-3604-4025-8cbc-05e07a27b62d admin admin] REQUEST_METHOD: `PUT` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:04.272350 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b687b-3604-4025-8cbc-05e07a27b62d admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:04.272652 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b687b-3604-4025-8cbc-05e07a27b62d admin admin] PATH_INFO: `/v3/projects/192b8e73733046a188480d27a9092022/users/e811744d2a884e119076cde1b96f4846/roles/5032636d8729461c9926a59b9bcd041b` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:04.286772 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b687b-3604-4025-8cbc-05e07a27b62d admin admin] RBAC: Authorizing `identity:create_grant(project_id=192b8e73733046a188480d27a9092022, user_id=e811744d2a884e119076cde1b96f4846, role_id=5032636d8729461c9926a59b9bcd041b)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:09:04.289704 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b687b-3604-4025-8cbc-05e07a27b62d admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:09:04.437673 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1431/2883] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:09:04 2022] PUT /identity/v3/projects/192b8e73733046a188480d27a9092022/users/e811744d2a884e119076cde1b96f4846/roles/5032636d8729461c9926a59b9bcd041b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:09:04.442561 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f20bd-f290-4782-accb-d00cce38f9d6 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:04.443374 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f20bd-f290-4782-accb-d00cce38f9d6 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:04.443651 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f20bd-f290-4782-accb-d00cce38f9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:04.462511 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-a10f20bd-f290-4782-accb-d00cce38f9d6 None None] MFA Rules not processed for user `795669ccac0f433b900784c76b5b0142`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:09:04.463718 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-a10f20bd-f290-4782-accb-d00cce38f9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:09:04.523352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1453/2884] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:09:04.533078 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b687b-3604-4025-8cbc-05e07a27b62d admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:09:04.534868 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-454b687b-3604-4025-8cbc-05e07a27b62d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:09:04.566347 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:09:04.593859 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b687b-3604-4025-8cbc-05e07a27b62d admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:09:04.597522 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96507cb1-c99f-4a55-bce0-fd878b92a3c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:09:04.598152 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96507cb1-c99f-4a55-bce0-fd878b92a3c0 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:04.598429 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96507cb1-c99f-4a55-bce0-fd878b92a3c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:04.598699 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96507cb1-c99f-4a55-bce0-fd878b92a3c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:04.608138 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96507cb1-c99f-4a55-bce0-fd878b92a3c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:09:04.609591 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96507cb1-c99f-4a55-bce0-fd878b92a3c0 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:09:04.619465 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1432/2885] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:09:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:09:10.603184 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc0135-523e-4364-9f1f-dca82df73833 None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:10.603677 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc0135-523e-4364-9f1f-dca82df73833 None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:10.604188 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc0135-523e-4364-9f1f-dca82df73833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:10.636189 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-85bc0135-523e-4364-9f1f-dca82df73833 None None] MFA Rules not processed for user `e811744d2a884e119076cde1b96f4846`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:09:10.637426 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-85bc0135-523e-4364-9f1f-dca82df73833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:09:10.709302 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1454/2886] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:09:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:09:10.714524 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96507cb1-c99f-4a55-bce0-fd878b92a3c0 service neutron] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:09:10.724262 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96507cb1-c99f-4a55-bce0-fd878b92a3c0 service neutron] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:09:10.728947 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae1fec-a9a7-4292-8ec7-7d774d33d8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:09:10.729553 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae1fec-a9a7-4292-8ec7-7d774d33d8b2 admin admin] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:10.729834 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae1fec-a9a7-4292-8ec7-7d774d33d8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:10.730108 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae1fec-a9a7-4292-8ec7-7d774d33d8b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:10.730738 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae1fec-a9a7-4292-8ec7-7d774d33d8b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:09:10.732668 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae1fec-a9a7-4292-8ec7-7d774d33d8b2 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:09:10.823140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1433/2887] 10.222.0.59 () {60 vars in 1153 bytes} [Fri Aug 26 01:09:10 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:09:10.827353 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc0135-523e-4364-9f1f-dca82df73833 None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:09:10.838706 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc0135-523e-4364-9f1f-dca82df73833 None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:09:10.853662 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:09:10.894693 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9ea2f-1acc-43e2-b678-0d68e4f41005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:09:10.895298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9ea2f-1acc-43e2-b678-0d68e4f41005 admin admin] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:10.895576 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9ea2f-1acc-43e2-b678-0d68e4f41005 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:10.895847 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9ea2f-1acc-43e2-b678-0d68e4f41005 admin admin] PATH_INFO: `/v3/users` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:10.896494 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f9ea2f-1acc-43e2-b678-0d68e4f41005 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:09:10.898406 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f9ea2f-1acc-43e2-b678-0d68e4f41005 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:09:11.001968 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1455/2888] 10.222.0.59 () {60 vars in 1144 bytes} [Fri Aug 26 01:09:10 2022] POST /identity/v3/users => generated 299 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 01:09:11.006002 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae1fec-a9a7-4292-8ec7-7d774d33d8b2 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:09:11.016953 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae1fec-a9a7-4292-8ec7-7d774d33d8b2 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:09:11.021504 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b218d23-522a-4c43-b719-f2beb9689a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:09:11.022099 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b218d23-522a-4c43-b719-f2beb9689a09 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:11.022380 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b218d23-522a-4c43-b719-f2beb9689a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:11.022649 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b218d23-522a-4c43-b719-f2beb9689a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:11.023280 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b218d23-522a-4c43-b719-f2beb9689a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:09:11.024995 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b218d23-522a-4c43-b719-f2beb9689a09 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:09:11.035489 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1434/2889] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 01:09:11.039900 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9ea2f-1acc-43e2-b678-0d68e4f41005 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:09:11.050233 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9ea2f-1acc-43e2-b678-0d68e4f41005 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:09:11.055527 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371fca32-fe25-4b97-89b6-792321708ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:09:11.056231 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fca32-fe25-4b97-89b6-792321708ca6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:11.056543 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fca32-fe25-4b97-89b6-792321708ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:11.056819 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fca32-fe25-4b97-89b6-792321708ca6 admin admin] PATH_INFO: `/v3/projects/409f95e72aab4ece80b05d9d89c28e3b/users/86a8077ce9a94b7daa66c980ba3af01c/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:11.071791 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371fca32-fe25-4b97-89b6-792321708ca6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=409f95e72aab4ece80b05d9d89c28e3b, user_id=86a8077ce9a94b7daa66c980ba3af01c, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:09:11.074460 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371fca32-fe25-4b97-89b6-792321708ca6 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:09:11.153128 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1456/2890] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:09:11 2022] PUT /identity/v3/projects/409f95e72aab4ece80b05d9d89c28e3b/users/86a8077ce9a94b7daa66c980ba3af01c/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:09:11.162272 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b218d23-522a-4c43-b719-f2beb9689a09 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:09:11.167694 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b218d23-522a-4c43-b719-f2beb9689a09 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:09:11.227094 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa5a63-16bd-4d40-99e2-818c7fd56429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:09:11.227711 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa5a63-16bd-4d40-99e2-818c7fd56429 admin admin] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:11.228129 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa5a63-16bd-4d40-99e2-818c7fd56429 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:11.228411 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa5a63-16bd-4d40-99e2-818c7fd56429 admin admin] PATH_INFO: `/v3/roles` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:11.229124 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa5a63-16bd-4d40-99e2-818c7fd56429 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:09:11.230737 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa5a63-16bd-4d40-99e2-818c7fd56429 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:09:11.240577 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1435/2891] 10.222.0.59 () {58 vars in 1122 bytes} [Fri Aug 26 01:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 01:09:11.245529 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371fca32-fe25-4b97-89b6-792321708ca6 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:09:11.256460 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371fca32-fe25-4b97-89b6-792321708ca6 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:09:11.261817 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399bdf7a-a76d-4cdc-9f47-20efcbfcbad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:09:11.262494 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399bdf7a-a76d-4cdc-9f47-20efcbfcbad9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:11.262776 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399bdf7a-a76d-4cdc-9f47-20efcbfcbad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:11.263051 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399bdf7a-a76d-4cdc-9f47-20efcbfcbad9 admin admin] PATH_INFO: `/v3/projects/409f95e72aab4ece80b05d9d89c28e3b/users/86a8077ce9a94b7daa66c980ba3af01c/roles/1a33e3c8cb8347feb9498595886cc621` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:11.265068 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399bdf7a-a76d-4cdc-9f47-20efcbfcbad9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=409f95e72aab4ece80b05d9d89c28e3b, user_id=86a8077ce9a94b7daa66c980ba3af01c, role_id=1a33e3c8cb8347feb9498595886cc621)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:09:11.267782 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399bdf7a-a76d-4cdc-9f47-20efcbfcbad9 admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:09:11.279014 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1457/2892] 10.222.0.59 () {60 vars in 1483 bytes} [Fri Aug 26 01:09:11 2022] PUT /identity/v3/projects/409f95e72aab4ece80b05d9d89c28e3b/users/86a8077ce9a94b7daa66c980ba3af01c/roles/1a33e3c8cb8347feb9498595886cc621 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:09:18.327371 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1b3c3-ff2b-45c8-b2b3-9175a3a55670 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:18.327944 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1b3c3-ff2b-45c8-b2b3-9175a3a55670 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:18.328142 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1b3c3-ff2b-45c8-b2b3-9175a3a55670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:18.363815 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-5cb1b3c3-ff2b-45c8-b2b3-9175a3a55670 None None] MFA Rules not processed for user `86a8077ce9a94b7daa66c980ba3af01c`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:09:18.365094 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-5cb1b3c3-ff2b-45c8-b2b3-9175a3a55670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:09:18.453318 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1436/2893] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:09:18 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:09:18.465466 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399bdf7a-a76d-4cdc-9f47-20efcbfcbad9 admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:09:18.467430 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-399bdf7a-a76d-4cdc-9f47-20efcbfcbad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:09:18.542650 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399bdf7a-a76d-4cdc-9f47-20efcbfcbad9 admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:09:18.546394 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4f63c-bb93-4be5-b473-5c3e12d98077 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abed7ebb0aae4e4693ce08e51a149312', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:09:18.549395 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4f63c-bb93-4be5-b473-5c3e12d98077 service glance] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:18.549395 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4f63c-bb93-4be5-b473-5c3e12d98077 service glance] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:18.549395 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4f63c-bb93-4be5-b473-5c3e12d98077 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:18.614546 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a4f63c-bb93-4be5-b473-5c3e12d98077 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:09:18.616055 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a4f63c-bb93-4be5-b473-5c3e12d98077 service glance] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:09:18.628061 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1458/2894] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:09:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 01:09:18.998432 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c765de6-1859-45c0-a9c9-14ab7519f915 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:19.001293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c765de6-1859-45c0-a9c9-14ab7519f915 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:19.001293 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c765de6-1859-45c0-a9c9-14ab7519f915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:19.027707 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:09:19.060728 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-7c765de6-1859-45c0-a9c9-14ab7519f915 None None] MFA Rules not processed for user `bbd9bbb083ab4f5e84e135ec5103cdae`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:09:19.062118 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-7c765de6-1859-45c0-a9c9-14ab7519f915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:09:19.128743 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1437/2895] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Aug 26 01:09:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:09:19.328634 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4f63c-bb93-4be5-b473-5c3e12d98077 service glance] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:09:19.330030 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-40a4f63c-bb93-4be5-b473-5c3e12d98077 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:09:19.356159 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:09:19.421633 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4f63c-bb93-4be5-b473-5c3e12d98077 service glance] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:09:19.426795 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cce637-4719-4f05-8003-5a1b9561cedf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f700502671446c797a566f07398a9fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:09:19.427411 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cce637-4719-4f05-8003-5a1b9561cedf service swift] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:19.427676 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cce637-4719-4f05-8003-5a1b9561cedf service swift] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:19.427947 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cce637-4719-4f05-8003-5a1b9561cedf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:19.436762 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cce637-4719-4f05-8003-5a1b9561cedf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:09:19.438485 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cce637-4719-4f05-8003-5a1b9561cedf service swift] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:09:19.447529 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1459/2896] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Aug 26 01:09:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 01:09:29.323873 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=36 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:09:29.457172 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d2fb9-9126-4d3b-93dd-b6788c8816b3 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:29.457461 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d2fb9-9126-4d3b-93dd-b6788c8816b3 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:29.457764 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d2fb9-9126-4d3b-93dd-b6788c8816b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:29.476978 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-a49d2fb9-9126-4d3b-93dd-b6788c8816b3 None None] MFA Rules not processed for user `e811744d2a884e119076cde1b96f4846`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:09:29.478180 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-a49d2fb9-9126-4d3b-93dd-b6788c8816b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:09:29.485407 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1438/2897] 10.222.0.59 () {58 vars in 958 bytes} [Fri Aug 26 01:09:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:09:29.527812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cce637-4719-4f05-8003-5a1b9561cedf service swift] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:09:29.537119 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cce637-4719-4f05-8003-5a1b9561cedf service swift] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:09:29.541465 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b09fb1-d2be-4d18-8810-89a081e32aa5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abed7ebb0aae4e4693ce08e51a149312', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:09:29.542067 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b09fb1-d2be-4d18-8810-89a081e32aa5 service glance] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:29.542347 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b09fb1-d2be-4d18-8810-89a081e32aa5 service glance] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:29.542626 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b09fb1-d2be-4d18-8810-89a081e32aa5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:29.544340 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-f0b09fb1-d2be-4d18-8810-89a081e32aa5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:09:29.614868 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b09fb1-d2be-4d18-8810-89a081e32aa5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:09:29.616321 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b09fb1-d2be-4d18-8810-89a081e32aa5 service glance] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:09:29.628276 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1460/2898] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:09:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:09:30.269450 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d2fb9-9126-4d3b-93dd-b6788c8816b3 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:09:30.330457 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d2fb9-9126-4d3b-93dd-b6788c8816b3 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:09:30.334204 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a89b4d-7ca4-4d54-9879-0cbb177a12c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:09:30.334827 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a89b4d-7ca4-4d54-9879-0cbb177a12c6 service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:30.335140 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a89b4d-7ca4-4d54-9879-0cbb177a12c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:30.336086 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a89b4d-7ca4-4d54-9879-0cbb177a12c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:30.338218 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-52a89b4d-7ca4-4d54-9879-0cbb177a12c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:09:30.352451 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a89b4d-7ca4-4d54-9879-0cbb177a12c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:09:30.354065 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a89b4d-7ca4-4d54-9879-0cbb177a12c6 service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:09:30.366235 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1439/2899] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:09:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:09:33.262726 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7792ab-a5b4-459f-8b92-d44feca5b32c None None] REQUEST_METHOD: `POST` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:33.263208 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7792ab-a5b4-459f-8b92-d44feca5b32c None None] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:33.263298 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7792ab-a5b4-459f-8b92-d44feca5b32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:33.283095 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.auth.core [None req-2e7792ab-a5b4-459f-8b92-d44feca5b32c None None] MFA Rules not processed for user `bbd9bbb083ab4f5e84e135ec5103cdae`. Rule list: `[]` (Enabled: `True`). {{(pid=88254) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:09:33.284302 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-2e7792ab-a5b4-459f-8b92-d44feca5b32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:09:33.292042 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1461/2900] 10.222.0.59 () {58 vars in 1009 bytes} [Fri Aug 26 01:09:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:09:33.304448 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a89b4d-7ca4-4d54-9879-0cbb177a12c6 service neutron] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:09:33.315851 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a89b4d-7ca4-4d54-9879-0cbb177a12c6 service neutron] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:09:33.321546 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda7e754-7280-42b3-9e97-10391ec4864c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f700502671446c797a566f07398a9fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:09:33.322228 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7e754-7280-42b3-9e97-10391ec4864c service swift] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:33.322516 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7e754-7280-42b3-9e97-10391ec4864c service swift] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:33.322789 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7e754-7280-42b3-9e97-10391ec4864c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:33.331544 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda7e754-7280-42b3-9e97-10391ec4864c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:09:33.333171 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda7e754-7280-42b3-9e97-10391ec4864c service swift] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:09:33.342315 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1440/2901] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Aug 26 01:09:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 01:09:41.288369 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7792ab-a5b4-459f-8b92-d44feca5b32c None None] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:09:41.290289 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-2e7792ab-a5b4-459f-8b92-d44feca5b32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:09:41.301605 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7792ab-a5b4-459f-8b92-d44feca5b32c None None] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:09:41.305607 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0c250-0892-4cae-acbf-92bf16960aa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:09:41.306201 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0c250-0892-4cae-acbf-92bf16960aa0 service neutron] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:41.306475 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0c250-0892-4cae-acbf-92bf16960aa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:41.306740 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0c250-0892-4cae-acbf-92bf16960aa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:41.308355 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-37f0c250-0892-4cae-acbf-92bf16960aa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:09:41.317967 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f0c250-0892-4cae-acbf-92bf16960aa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:09:41.322180 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f0c250-0892-4cae-acbf-92bf16960aa0 service neutron] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:09:41.327562 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1462/2902] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:09:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:09:42.053515 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda7e754-7280-42b3-9e97-10391ec4864c service swift] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:09:42.054943 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-bda7e754-7280-42b3-9e97-10391ec4864c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:09:42.071815 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda7e754-7280-42b3-9e97-10391ec4864c service swift] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:09:42.077710 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3251f4-4de1-4df5-be44-e99c8a4aee1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:09:42.078640 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3251f4-4de1-4df5-be44-e99c8a4aee1d service nova] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:42.078640 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3251f4-4de1-4df5-be44-e99c8a4aee1d service nova] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:42.078883 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3251f4-4de1-4df5-be44-e99c8a4aee1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:42.080639 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-9c3251f4-4de1-4df5-be44-e99c8a4aee1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:09:42.094089 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3251f4-4de1-4df5-be44-e99c8a4aee1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:09:42.095782 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3251f4-4de1-4df5-be44-e99c8a4aee1d service nova] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:09:42.106616 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1441/2903] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:09:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:09:47.490461 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0c250-0892-4cae-acbf-92bf16960aa0 service neutron] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:09:47.504739 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0c250-0892-4cae-acbf-92bf16960aa0 service neutron] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:09:47.510774 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64899de2-87da-405e-bb7d-c36b72134945 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:09:47.511375 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64899de2-87da-405e-bb7d-c36b72134945 service nova] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:09:47.511653 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64899de2-87da-405e-bb7d-c36b72134945 service nova] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:09:47.511924 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64899de2-87da-405e-bb7d-c36b72134945 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:09:47.513656 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-64899de2-87da-405e-bb7d-c36b72134945 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:09:47.528992 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64899de2-87da-405e-bb7d-c36b72134945 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:09:47.530571 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64899de2-87da-405e-bb7d-c36b72134945 service nova] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:09:47.542337 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1463/2904] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:09:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:09:52.101982 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=41 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:09:57.536714 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=80 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:10:00.330613 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3251f4-4de1-4df5-be44-e99c8a4aee1d service nova] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:10:00.340294 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3251f4-4de1-4df5-be44-e99c8a4aee1d service nova] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:10:00.343902 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755d41ee-d1ac-4c71-a6f5-268164ef838f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4698a49ed4a04f58a488359a2742e5da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:10:00.344522 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d41ee-d1ac-4c71-a6f5-268164ef838f service neutron] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:10:00.344895 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d41ee-d1ac-4c71-a6f5-268164ef838f service neutron] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:10:00.345169 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d41ee-d1ac-4c71-a6f5-268164ef838f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:10:00.354223 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755d41ee-d1ac-4c71-a6f5-268164ef838f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:10:00.355639 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755d41ee-d1ac-4c71-a6f5-268164ef838f service neutron] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:10:00.364816 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1442/2905] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:10:00 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:10:10.360795 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=6 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:11:09.869125 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64899de2-87da-405e-bb7d-c36b72134945 service nova] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:11:09.871459 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-64899de2-87da-405e-bb7d-c36b72134945 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:11:09.898474 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64899de2-87da-405e-bb7d-c36b72134945 service nova] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:11:09.905724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a58353-887f-4c43-b5c7-25bc33512f7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0602593263ca4b05a657d5b7697f932f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:11:09.905724 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a58353-887f-4c43-b5c7-25bc33512f7e service nova] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:11:09.907515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a58353-887f-4c43-b5c7-25bc33512f7e service nova] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:11:09.907515 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a58353-887f-4c43-b5c7-25bc33512f7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:11:09.909179 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-b4a58353-887f-4c43-b5c7-25bc33512f7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:11:09.925305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a58353-887f-4c43-b5c7-25bc33512f7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:11:09.927350 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a58353-887f-4c43-b5c7-25bc33512f7e service nova] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:11:09.939447 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1464/2906] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:11:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:11:19.934313 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=6 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:11:20.067901 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221af9ca-a273-4fb4-85fc-9005a6a77be6 None None] REQUEST_METHOD: `POST` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:11:20.068378 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221af9ca-a273-4fb4-85fc-9005a6a77be6 None None] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:11:20.068530 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221af9ca-a273-4fb4-85fc-9005a6a77be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:11:20.087966 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.auth.core [None req-221af9ca-a273-4fb4-85fc-9005a6a77be6 None None] MFA Rules not processed for user `bbd9bbb083ab4f5e84e135ec5103cdae`. Rule list: `[]` (Enabled: `True`). {{(pid=88253) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 01:11:20.089264 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-221af9ca-a273-4fb4-85fc-9005a6a77be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:11:20.096613 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1443/2907] 10.222.0.59 () {58 vars in 967 bytes} [Fri Aug 26 01:11:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 01:11:20.137065 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a58353-887f-4c43-b5c7-25bc33512f7e service nova] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:11:20.148743 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a58353-887f-4c43-b5c7-25bc33512f7e service nova] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:11:20.154524 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65819d5-3eba-475a-ae20-b46d8009912f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f700502671446c797a566f07398a9fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:11:20.155127 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65819d5-3eba-475a-ae20-b46d8009912f service swift] REQUEST_METHOD: `GET` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:11:20.155405 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65819d5-3eba-475a-ae20-b46d8009912f service swift] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:11:20.155680 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65819d5-3eba-475a-ae20-b46d8009912f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:11:20.163946 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65819d5-3eba-475a-ae20-b46d8009912f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:11:20.166108 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65819d5-3eba-475a-ae20-b46d8009912f service swift] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:11:20.174108 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1465/2908] 10.222.0.59 () {60 vars in 1384 bytes} [Fri Aug 26 01:11:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 01:11:28.753637 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221af9ca-a273-4fb4-85fc-9005a6a77be6 None None] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:11:28.819033 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221af9ca-a273-4fb4-85fc-9005a6a77be6 None None] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:11:28.824022 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89f485-aaac-4eda-83b1-3a5820bcdb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:11:28.824680 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89f485-aaac-4eda-83b1-3a5820bcdb4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:11:28.824968 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89f485-aaac-4eda-83b1-3a5820bcdb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:11:28.825239 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89f485-aaac-4eda-83b1-3a5820bcdb4e admin admin] PATH_INFO: `/v3/users/e811744d2a884e119076cde1b96f4846` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:11:28.826403 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd89f485-aaac-4eda-83b1-3a5820bcdb4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e811744d2a884e119076cde1b96f4846)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:11:28.828308 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd89f485-aaac-4eda-83b1-3a5820bcdb4e admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:11:29.038228 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-dd89f485-aaac-4eda-83b1-3a5820bcdb4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e811744d2a884e119076cde1b96f4846'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:11:29.050014 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-dd89f485-aaac-4eda-83b1-3a5820bcdb4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e811744d2a884e119076cde1b96f4846'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:11:29.151256 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-dd89f485-aaac-4eda-83b1-3a5820bcdb4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e811744d2a884e119076cde1b96f4846'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:11:29.151876 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-dd89f485-aaac-4eda-83b1-3a5820bcdb4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e811744d2a884e119076cde1b96f4846'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:11:29.152510 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-dd89f485-aaac-4eda-83b1-3a5820bcdb4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e811744d2a884e119076cde1b96f4846'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:11:29.165453 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1444/2909] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:11:28 2022] DELETE /identity/v3/users/e811744d2a884e119076cde1b96f4846 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:11:29.414749 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65819d5-3eba-475a-ae20-b46d8009912f service swift] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:11:29.427866 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65819d5-3eba-475a-ae20-b46d8009912f service swift] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:11:29.487916 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea288743-f690-4e37-8df5-a236601d4bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:11:29.488615 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea288743-f690-4e37-8df5-a236601d4bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:11:29.488899 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea288743-f690-4e37-8df5-a236601d4bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:11:29.489173 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea288743-f690-4e37-8df5-a236601d4bfb admin admin] PATH_INFO: `/v3/projects/192b8e73733046a188480d27a9092022` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:11:29.490305 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea288743-f690-4e37-8df5-a236601d4bfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=192b8e73733046a188480d27a9092022)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:11:29.492215 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea288743-f690-4e37-8df5-a236601d4bfb admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:11:29.636114 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ea288743-f690-4e37-8df5-a236601d4bfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '192b8e73733046a188480d27a9092022', 'request_id': 'req-ea288743-f690-4e37-8df5-a236601d4bfb'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:11:29.724843 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ea288743-f690-4e37-8df5-a236601d4bfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '192b8e73733046a188480d27a9092022', 'request_id': 'req-ea288743-f690-4e37-8df5-a236601d4bfb'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:11:29.731470 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ea288743-f690-4e37-8df5-a236601d4bfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '192b8e73733046a188480d27a9092022', 'request_id': 'req-ea288743-f690-4e37-8df5-a236601d4bfb'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:11:29.744056 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ea288743-f690-4e37-8df5-a236601d4bfb admin admin] The token cache is being invalidate because project 192b8e73733046a188480d27a9092022 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:11:29.744352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ea288743-f690-4e37-8df5-a236601d4bfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:11:29.745011 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-ea288743-f690-4e37-8df5-a236601d4bfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:11:29.746580 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1466/2910] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:11:29 2022] DELETE /identity/v3/projects/192b8e73733046a188480d27a9092022 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:11:29.750561 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89f485-aaac-4eda-83b1-3a5820bcdb4e admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:11:29.752053 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-dd89f485-aaac-4eda-83b1-3a5820bcdb4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:11:29.764998 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89f485-aaac-4eda-83b1-3a5820bcdb4e admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:11:29.820757 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c77d64-3d99-43f7-97f9-086c05689a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:11:29.821386 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c77d64-3d99-43f7-97f9-086c05689a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:11:29.821669 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c77d64-3d99-43f7-97f9-086c05689a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:11:29.821946 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c77d64-3d99-43f7-97f9-086c05689a61 admin admin] PATH_INFO: `/v3/users/86a8077ce9a94b7daa66c980ba3af01c` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:11:29.823082 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c77d64-3d99-43f7-97f9-086c05689a61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86a8077ce9a94b7daa66c980ba3af01c)` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:11:29.825033 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c77d64-3d99-43f7-97f9-086c05689a61 admin admin] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:11:29.838625 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:11:30.052222 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-98c77d64-3d99-43f7-97f9-086c05689a61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86a8077ce9a94b7daa66c980ba3af01c'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:11:30.064051 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-98c77d64-3d99-43f7-97f9-086c05689a61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86a8077ce9a94b7daa66c980ba3af01c'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:11:30.153651 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-98c77d64-3d99-43f7-97f9-086c05689a61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86a8077ce9a94b7daa66c980ba3af01c'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:11:30.154303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-98c77d64-3d99-43f7-97f9-086c05689a61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86a8077ce9a94b7daa66c980ba3af01c'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:11:30.154902 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.notifications [None req-98c77d64-3d99-43f7-97f9-086c05689a61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86a8077ce9a94b7daa66c980ba3af01c'} {{(pid=88253) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:11:30.170705 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1445/2911] 10.222.0.59 () {58 vars in 1224 bytes} [Fri Aug 26 01:11:29 2022] DELETE /identity/v3/users/86a8077ce9a94b7daa66c980ba3af01c => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:11:30.415812 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea288743-f690-4e37-8df5-a236601d4bfb admin admin] Authenticating user token {{(pid=88254) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:11:30.417273 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.fernet_utils [None req-ea288743-f690-4e37-8df5-a236601d4bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88254) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:11:30.435252 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea288743-f690-4e37-8df5-a236601d4bfb admin admin] Validating token access rules against request {{(pid=88254) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:11:30.490714 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c9321-4f92-4f56-949f-7f321e2883fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '795669ccac0f433b900784c76b5b0142', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86ade54a72a44599b7a6313f909a3551', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88254) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:11:30.491352 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c9321-4f92-4f56-949f-7f321e2883fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:11:30.491637 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c9321-4f92-4f56-949f-7f321e2883fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:11:30.491912 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c9321-4f92-4f56-949f-7f321e2883fb admin admin] PATH_INFO: `/v3/projects/409f95e72aab4ece80b05d9d89c28e3b` {{(pid=88254) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:11:30.493658 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8c9321-4f92-4f56-949f-7f321e2883fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=409f95e72aab4ece80b05d9d89c28e3b)` {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:11:30.495637 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8c9321-4f92-4f56-949f-7f321e2883fb admin admin] RBAC: Authorization granted {{(pid=88254) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:11:30.516595 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:11:30.606772 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-df8c9321-4f92-4f56-949f-7f321e2883fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '409f95e72aab4ece80b05d9d89c28e3b', 'request_id': 'req-df8c9321-4f92-4f56-949f-7f321e2883fb'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:11:30.699693 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-df8c9321-4f92-4f56-949f-7f321e2883fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '409f95e72aab4ece80b05d9d89c28e3b', 'request_id': 'req-df8c9321-4f92-4f56-949f-7f321e2883fb'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:11:30.705743 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-df8c9321-4f92-4f56-949f-7f321e2883fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '409f95e72aab4ece80b05d9d89c28e3b', 'request_id': 'req-df8c9321-4f92-4f56-949f-7f321e2883fb'} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:11:30.717346 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-df8c9321-4f92-4f56-949f-7f321e2883fb admin admin] The token cache is being invalidate because project 409f95e72aab4ece80b05d9d89c28e3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88254) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 01:11:30.717632 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-df8c9321-4f92-4f56-949f-7f321e2883fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:11:30.718265 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG keystone.notifications [None req-df8c9321-4f92-4f56-949f-7f321e2883fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88254) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 01:11:30.719731 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: [pid: 88254|app: 0|req: 1467/2912] 10.222.0.59 () {58 vars in 1233 bytes} [Fri Aug 26 01:11:30 2022] DELETE /identity/v3/projects/409f95e72aab4ece80b05d9d89c28e3b => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 01:11:40.167098 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=21,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:11:40.784663 dv-cinder-835384-25-iscsi devstack@keystone.service[88254]: DEBUG dbcounter [-] [88254] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88254) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 01:12:05.501583 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c77d64-3d99-43f7-97f9-086c05689a61 admin admin] Authenticating user token {{(pid=88253) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 01:12:05.503018 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-98c77d64-3d99-43f7-97f9-086c05689a61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:12:05.639470 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c77d64-3d99-43f7-97f9-086c05689a61 admin admin] Validating token access rules against request {{(pid=88253) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 01:12:05.697492 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ee491-3c7d-45e3-b2a5-33fb61426a24 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2876f4182f0949039b862f053533fe85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fde7ac3ed472433ab3bba56e5e091510', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88253) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 01:12:05.698023 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ee491-3c7d-45e3-b2a5-33fb61426a24 service placement] REQUEST_METHOD: `GET` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 01:12:05.698303 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ee491-3c7d-45e3-b2a5-33fb61426a24 service placement] SCRIPT_NAME: `/identity` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 01:12:05.698578 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ee491-3c7d-45e3-b2a5-33fb61426a24 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88253) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 01:12:05.700297 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.fernet_utils [None req-000ee491-3c7d-45e3-b2a5-33fb61426a24 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88253) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 01:12:05.711405 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000ee491-3c7d-45e3-b2a5-33fb61426a24 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 01:12:05.713015 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000ee491-3c7d-45e3-b2a5-33fb61426a24 service placement] RBAC: Authorization granted {{(pid=88253) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 01:12:05.723150 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: [pid: 88253|app: 0|req: 1446/2913] 10.222.0.59 () {60 vars in 1365 bytes} [Fri Aug 26 01:12:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 01:12:15.719074 dv-cinder-835384-25-iscsi devstack@keystone.service[88253]: DEBUG dbcounter [-] [88253] Writing DB stats keystone:SELECT=89 {{(pid=88253) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}